Jul 07 18:49:50.907294 ip-10-0-142-66 systemd[1]: Starting Kubernetes Kubelet... Jul 07 18:49:51.345831 ip-10-0-142-66 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 18:49:51.345831 ip-10-0-142-66 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 18:49:51.345831 ip-10-0-142-66 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 18:49:51.345831 ip-10-0-142-66 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 18:49:51.345831 ip-10-0-142-66 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 18:49:51.348441 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.348366 2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 18:49:51.356529 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356506 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:49:51.356529 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356526 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:49:51.356589 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356531 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:49:51.356589 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356534 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:49:51.356589 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356537 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:49:51.356589 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356540 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:49:51.356589 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356543 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:49:51.356589 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356546 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:49:51.356589 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356548 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:49:51.356589 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356551 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:49:51.356589 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356553 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:49:51.356589 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356556 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:49:51.356589 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356558 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:49:51.356589 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356561 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:49:51.356589 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356563 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:49:51.356589 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356566 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:49:51.356589 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356568 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:49:51.356589 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356571 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:49:51.356589 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356573 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:49:51.356589 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356576 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:49:51.356589 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356579 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:49:51.356589 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356582 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:49:51.357039 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356585 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:49:51.357039 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356587 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:49:51.357039 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356591 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:49:51.357039 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356596 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:49:51.357039 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356599 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:49:51.357039 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356601 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:49:51.357039 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356604 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:49:51.357039 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356607 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:49:51.357039 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356609 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:49:51.357039 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356611 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:49:51.357039 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356614 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:49:51.357039 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356616 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:49:51.357039 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356619 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:49:51.357039 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356628 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:49:51.357039 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356632 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:49:51.357039 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356634 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:49:51.357039 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356637 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:49:51.357039 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356640 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:49:51.357039 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356642 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:49:51.357496 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356645 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:49:51.357496 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356647 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:49:51.357496 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356649 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:49:51.357496 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356651 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:49:51.357496 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356654 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:49:51.357496 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356656 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:49:51.357496 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356658 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:49:51.357496 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356661 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:49:51.357496 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356663 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:49:51.357496 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356666 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:49:51.357496 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356668 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:49:51.357496 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356670 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:49:51.357496 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356673 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:49:51.357496 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356676 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:49:51.357496 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356678 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:49:51.357496 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356681 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:49:51.357496 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356684 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:49:51.357496 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356687 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:49:51.357496 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356689 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:49:51.357496 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356692 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:49:51.357966 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356694 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:49:51.357966 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356698 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:49:51.357966 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356701 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:49:51.357966 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356703 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:49:51.357966 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356706 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:49:51.357966 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356709 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:49:51.357966 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356711 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:49:51.357966 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356714 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:49:51.357966 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356716 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:49:51.357966 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356719 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:49:51.357966 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356722 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:49:51.357966 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356724 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:49:51.357966 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356726 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:49:51.357966 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356729 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:49:51.357966 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356731 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:49:51.357966 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356733 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:49:51.357966 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356736 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:49:51.357966 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356738 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:49:51.357966 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356740 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:49:51.358405 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356742 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:49:51.358405 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356745 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:49:51.358405 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356747 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:49:51.358405 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356749 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:49:51.358405 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356752 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:49:51.358405 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356754 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:49:51.358405 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356757 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:49:51.358405 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356760 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:49:51.358405 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356763 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:49:51.358405 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356765 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:49:51.358405 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356767 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:49:51.358405 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356770 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:49:51.358405 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356772 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:49:51.358405 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356774 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:49:51.358405 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356776 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:49:51.358405 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356779 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:49:51.358405 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356781 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:49:51.358405 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356783 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:49:51.358405 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.356786 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:49:51.358405 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357445 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:49:51.358861 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357452 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:49:51.358861 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357455 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:49:51.358861 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357457 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:49:51.358861 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357460 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:49:51.358861 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357463 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:49:51.358861 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357466 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:49:51.358861 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357486 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:49:51.358861 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357490 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:49:51.358861 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357493 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:49:51.358861 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357496 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:49:51.358861 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357499 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:49:51.358861 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357502 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:49:51.358861 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357504 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:49:51.358861 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357507 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:49:51.358861 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357510 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:49:51.358861 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357513 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:49:51.358861 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357516 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:49:51.358861 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357519 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:49:51.358861 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357521 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:49:51.359340 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357524 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:49:51.359340 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357527 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:49:51.359340 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357530 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:49:51.359340 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357532 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:49:51.359340 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357535 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:49:51.359340 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357538 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:49:51.359340 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357541 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:49:51.359340 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357543 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:49:51.359340 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357547 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:49:51.359340 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357550 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:49:51.359340 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357552 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:49:51.359340 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357555 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:49:51.359340 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357558 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:49:51.359340 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357561 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:49:51.359340 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357564 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:49:51.359340 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357566 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:49:51.359340 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357568 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:49:51.359340 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357571 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:49:51.359340 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357574 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:49:51.359785 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357577 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:49:51.359785 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357579 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:49:51.359785 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357581 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:49:51.359785 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357584 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:49:51.359785 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357586 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:49:51.359785 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357588 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:49:51.359785 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357591 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:49:51.359785 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357593 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:49:51.359785 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357595 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:49:51.359785 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357598 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:49:51.359785 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357600 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:49:51.359785 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357603 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:49:51.359785 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357606 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:49:51.359785 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357609 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:49:51.359785 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357613 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:49:51.359785 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357616 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:49:51.359785 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357619 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:49:51.359785 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357621 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:49:51.359785 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357624 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:49:51.359785 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357626 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:49:51.360363 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357628 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:49:51.360363 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357631 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:49:51.360363 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357633 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:49:51.360363 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357636 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:49:51.360363 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357638 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:49:51.360363 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357641 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:49:51.360363 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357643 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:49:51.360363 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357646 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:49:51.360363 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357648 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:49:51.360363 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357650 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:49:51.360363 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357654 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:49:51.360363 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357656 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:49:51.360363 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357659 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:49:51.360363 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357661 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:49:51.360363 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357664 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:49:51.360363 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357666 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:49:51.360363 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357669 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:49:51.360363 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357672 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:49:51.360363 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357674 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:49:51.360363 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357676 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:49:51.360838 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357678 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:49:51.360838 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357681 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:49:51.360838 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357683 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:49:51.360838 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357686 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:49:51.360838 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357688 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:49:51.360838 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357690 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:49:51.360838 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357692 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:49:51.360838 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357695 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:49:51.360838 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357697 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:49:51.360838 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357699 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:49:51.360838 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357702 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:49:51.360838 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357704 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:49:51.360838 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357707 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:49:51.360838 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357709 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:49:51.360838 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357711 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:49:51.360838 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357714 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:49:51.360838 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357716 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:49:51.360838 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357718 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:49:51.360838 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357720 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:49:51.360838 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.357722 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:49:51.361309 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358077 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:49:51.361309 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358083 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:49:51.361309 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358085 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:49:51.361309 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358088 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:49:51.361309 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358092 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:49:51.361309 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358094 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:49:51.361309 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358097 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:49:51.361309 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358099 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:49:51.361309 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358102 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:49:51.361309 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358105 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:49:51.361309 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358107 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:49:51.361309 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358110 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:49:51.361309 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358112 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:49:51.361309 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358115 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:49:51.361309 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358117 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:49:51.361309 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358120 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:49:51.361309 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358122 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:49:51.361309 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358125 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:49:51.361309 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358127 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:49:51.361309 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358130 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:49:51.361777 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358133 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:49:51.361777 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358135 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:49:51.361777 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358138 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:49:51.361777 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358140 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:49:51.361777 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358142 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:49:51.361777 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358145 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:49:51.361777 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358147 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:49:51.361777 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358149 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:49:51.361777 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358152 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:49:51.361777 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358154 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:49:51.361777 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358156 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:49:51.361777 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358159 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:49:51.361777 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358161 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:49:51.361777 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358164 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:49:51.361777 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358166 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:49:51.361777 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358168 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:49:51.361777 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358171 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:49:51.361777 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358173 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:49:51.361777 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358176 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:49:51.361777 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358178 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:49:51.362270 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358183 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:49:51.362270 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358186 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:49:51.362270 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358189 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:49:51.362270 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358192 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:49:51.362270 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358194 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:49:51.362270 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358196 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:49:51.362270 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358199 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:49:51.362270 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358202 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:49:51.362270 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358204 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:49:51.362270 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358207 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:49:51.362270 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358209 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:49:51.362270 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358212 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:49:51.362270 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358214 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:49:51.362270 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358216 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:49:51.362270 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358220 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:49:51.362270 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358223 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:49:51.362270 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358226 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:49:51.362270 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358229 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:49:51.362270 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358233 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:49:51.362707 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358237 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:49:51.362707 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358239 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:49:51.362707 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358241 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:49:51.362707 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358244 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:49:51.362707 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358246 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:49:51.362707 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358248 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:49:51.362707 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358251 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:49:51.362707 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358254 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:49:51.362707 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358256 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:49:51.362707 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358263 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:49:51.362707 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358266 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:49:51.362707 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358269 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:49:51.362707 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358271 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:49:51.362707 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358274 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:49:51.362707 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358276 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:49:51.362707 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358278 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:49:51.362707 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358281 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:49:51.362707 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358284 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:49:51.362707 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358286 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:49:51.362707 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358288 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:49:51.363187 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358291 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:49:51.363187 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358293 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:49:51.363187 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358296 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:49:51.363187 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358298 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:49:51.363187 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358300 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:49:51.363187 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358303 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:49:51.363187 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358305 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:49:51.363187 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358307 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:49:51.363187 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358309 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:49:51.363187 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358312 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:49:51.363187 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358314 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:49:51.363187 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358316 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:49:51.363187 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358319 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:49:51.363187 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358321 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:49:51.363187 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358323 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:49:51.363187 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358325 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:49:51.363187 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358328 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:49:51.363187 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358330 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:49:51.363187 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358332 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:49:51.363187 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358334 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358391 2564 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358397 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358403 2564 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358407 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358411 2564 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358417 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358420 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358424 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358427 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358429 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358435 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358438 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358441 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358444 2564 flags.go:64] FLAG: --cgroup-root="" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358446 2564 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358449 2564 flags.go:64] FLAG: --client-ca-file="" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358452 2564 flags.go:64] FLAG: --cloud-provider="external" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358454 2564 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358458 2564 flags.go:64] FLAG: --cluster-domain="" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358460 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358463 2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358466 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358469 2564 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 18:49:51.363673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358472 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358475 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358478 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358481 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358483 2564 flags.go:64] FLAG: --contention-profiling="false" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358486 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358489 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358491 2564 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358494 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358497 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358500 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358503 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358505 2564 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358508 2564 flags.go:64] FLAG: --enable-server="true" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358512 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358516 2564 flags.go:64] FLAG: --event-burst="100" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358519 2564 flags.go:64] FLAG: --event-qps="50" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358522 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358524 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358530 2564 flags.go:64] FLAG: --eviction-hard="" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358533 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358535 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358538 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358541 2564 flags.go:64] FLAG: --eviction-soft="" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358543 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 18:49:51.364242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358546 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358548 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358551 2564 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358554 2564 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358556 2564 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358559 2564 flags.go:64] FLAG: --feature-gates="" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358562 2564 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358565 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358567 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358570 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358573 2564 flags.go:64] FLAG: --healthz-port="10248" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358575 2564 flags.go:64] FLAG: --help="false" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358578 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-142-66.ec2.internal" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358581 2564 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358583 2564 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358586 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358589 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358592 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358595 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358597 2564 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358600 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358603 2564 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358607 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358609 2564 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 18:49:51.364870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358612 2564 flags.go:64] FLAG: --kube-reserved="" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358614 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358618 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358621 2564 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358623 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358626 2564 flags.go:64] FLAG: --lock-file="" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358629 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358631 2564 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358634 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358639 2564 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358642 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358644 2564 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358647 2564 flags.go:64] FLAG: --logging-format="text" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358649 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358652 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358655 2564 flags.go:64] FLAG: --manifest-url="" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358657 2564 flags.go:64] FLAG: --manifest-url-header="" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358661 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358664 2564 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358667 2564 flags.go:64] FLAG: --max-pods="110" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358670 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358672 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358675 2564 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358677 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358680 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 18:49:51.365448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358683 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358685 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358691 2564 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358694 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358698 2564 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358701 2564 flags.go:64] FLAG: --pod-cidr="" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358704 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358709 2564 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358712 2564 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358714 2564 flags.go:64] FLAG: --pods-per-core="0" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358719 2564 flags.go:64] FLAG: --port="10250" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358722 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358725 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-035605102e0a32780" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358727 2564 flags.go:64] FLAG: --qos-reserved="" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358730 2564 flags.go:64] FLAG: --read-only-port="10255" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358733 2564 flags.go:64] FLAG: --register-node="true" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358735 2564 flags.go:64] FLAG: --register-with-taints="" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358738 2564 flags.go:64] FLAG: --registry-burst="10" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358741 2564 flags.go:64] FLAG: --registry-qps="5" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358743 2564 flags.go:64] FLAG: --reserved-cpus="" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358746 2564 flags.go:64] FLAG: --reserved-memory="" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358749 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358752 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358754 2564 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358757 2564 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 18:49:51.366021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358759 2564 flags.go:64] FLAG: --runonce="false" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358762 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358764 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358767 2564 flags.go:64] FLAG: --seccomp-default="false" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358770 2564 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358772 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358775 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358777 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358780 2564 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358783 2564 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358785 2564 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358789 2564 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358791 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358794 2564 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358798 2564 flags.go:64] FLAG: --system-cgroups="" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358800 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358806 2564 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358809 2564 flags.go:64] FLAG: --tls-cert-file="" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358812 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358815 2564 flags.go:64] FLAG: --tls-min-version="" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358817 2564 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358820 2564 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358823 2564 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358825 2564 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358828 2564 flags.go:64] FLAG: --v="2" Jul 07 18:49:51.366626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358831 2564 flags.go:64] FLAG: --version="false" Jul 07 18:49:51.367235 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358835 2564 flags.go:64] FLAG: --vmodule="" Jul 07 18:49:51.367235 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358838 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 18:49:51.367235 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.358841 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 18:49:51.367235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358941 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:49:51.367235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358946 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:49:51.367235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358949 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:49:51.367235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358952 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:49:51.367235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358955 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:49:51.367235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358957 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:49:51.367235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358960 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:49:51.367235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358962 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:49:51.367235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358965 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:49:51.367235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358967 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:49:51.367235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358969 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:49:51.367235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358972 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:49:51.367235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358974 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:49:51.367235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358977 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:49:51.367235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358980 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:49:51.367235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358983 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:49:51.367235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358985 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:49:51.367673 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358988 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:49:51.367673 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358991 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:49:51.367673 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358993 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:49:51.367673 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358996 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:49:51.367673 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.358999 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:49:51.367673 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359001 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:49:51.367673 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359003 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:49:51.367673 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359006 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:49:51.367673 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359008 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:49:51.367673 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359010 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:49:51.367673 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359013 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:49:51.367673 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359015 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:49:51.367673 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359019 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:49:51.367673 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359022 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:49:51.367673 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359025 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:49:51.367673 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359028 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:49:51.367673 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359030 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:49:51.367673 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359033 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:49:51.367673 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359036 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:49:51.368235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359038 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:49:51.368235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359041 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:49:51.368235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359059 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:49:51.368235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359061 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:49:51.368235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359064 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:49:51.368235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359067 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:49:51.368235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359069 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:49:51.368235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359072 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:49:51.368235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359075 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:49:51.368235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359078 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:49:51.368235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359082 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:49:51.368235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359085 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:49:51.368235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359087 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:49:51.368235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359090 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:49:51.368235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359092 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:49:51.368235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359095 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:49:51.368235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359098 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:49:51.368235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359101 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:49:51.368235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359103 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:49:51.368696 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359106 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:49:51.368696 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359108 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:49:51.368696 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359110 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:49:51.368696 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359113 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:49:51.368696 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359115 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:49:51.368696 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359118 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:49:51.368696 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359120 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:49:51.368696 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359123 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:49:51.368696 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359125 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:49:51.368696 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359128 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:49:51.368696 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359130 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:49:51.368696 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359133 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:49:51.368696 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359135 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:49:51.368696 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359137 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:49:51.368696 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359139 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:49:51.368696 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359142 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:49:51.368696 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359144 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:49:51.368696 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359147 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:49:51.368696 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359149 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:49:51.368696 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359152 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:49:51.369432 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359154 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:49:51.369432 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359156 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:49:51.369432 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359159 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:49:51.369432 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359162 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:49:51.369432 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359164 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:49:51.369432 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359167 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:49:51.369432 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359169 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:49:51.369432 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359172 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:49:51.369432 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359175 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:49:51.369432 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359180 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:49:51.369432 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359182 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:49:51.369432 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359185 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:49:51.369432 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359188 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:49:51.369432 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359190 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:49:51.369432 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359193 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:49:51.369432 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359195 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:49:51.369432 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359197 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:49:51.369432 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359199 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:49:51.369432 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359202 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:49:51.369432 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359204 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:49:51.369956 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359207 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:49:51.369956 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359209 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:49:51.369956 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359211 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:49:51.369956 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.359214 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:49:51.369956 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.359222 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 18:49:51.369956 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.366015 2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 18:49:51.369956 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.366029 2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 18:49:51.369956 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.366063 2564 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 18:49:51.369956 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.366068 2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 18:49:51.369956 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366116 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:49:51.369956 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366121 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:49:51.369956 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366124 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:49:51.369956 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366127 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:49:51.369956 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366130 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:49:51.369956 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366132 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:49:51.370376 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366137 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:49:51.370376 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366140 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:49:51.370376 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366143 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:49:51.370376 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366146 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:49:51.370376 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366148 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:49:51.370376 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366151 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:49:51.370376 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366153 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:49:51.370376 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366156 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:49:51.370376 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366158 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:49:51.370376 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366161 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:49:51.370376 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366163 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:49:51.370376 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366166 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:49:51.370376 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366169 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:49:51.370376 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366172 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:49:51.370376 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366175 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:49:51.370376 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366177 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:49:51.370376 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366180 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:49:51.370376 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366182 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:49:51.370376 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366184 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:49:51.370831 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366187 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:49:51.370831 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366189 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:49:51.370831 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366192 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:49:51.370831 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366194 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:49:51.370831 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366198 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:49:51.370831 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366201 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:49:51.370831 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366203 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:49:51.370831 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366205 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:49:51.370831 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366208 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:49:51.370831 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366210 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:49:51.370831 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366212 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:49:51.370831 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366215 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:49:51.370831 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366217 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:49:51.370831 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366220 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:49:51.370831 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366222 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:49:51.370831 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366224 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:49:51.370831 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366226 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:49:51.370831 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366229 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:49:51.370831 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366231 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:49:51.370831 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366234 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:49:51.371368 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366236 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:49:51.371368 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366238 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:49:51.371368 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366241 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:49:51.371368 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366243 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:49:51.371368 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366245 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:49:51.371368 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366248 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:49:51.371368 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366250 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:49:51.371368 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366252 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:49:51.371368 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366255 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:49:51.371368 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366257 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:49:51.371368 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366260 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:49:51.371368 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366262 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:49:51.371368 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366264 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:49:51.371368 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366267 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:49:51.371368 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366269 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:49:51.371368 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366272 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:49:51.371368 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366275 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:49:51.371368 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366277 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:49:51.371368 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366280 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:49:51.371368 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366282 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:49:51.371827 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366284 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:49:51.371827 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366287 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:49:51.371827 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366289 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:49:51.371827 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366291 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:49:51.371827 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366293 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:49:51.371827 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366296 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:49:51.371827 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366298 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:49:51.371827 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366300 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:49:51.371827 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366303 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:49:51.371827 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366305 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:49:51.371827 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366307 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:49:51.371827 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366309 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:49:51.371827 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366312 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:49:51.371827 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366314 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:49:51.371827 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366317 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:49:51.371827 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366319 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:49:51.371827 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366321 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:49:51.371827 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366323 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:49:51.371827 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366326 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:49:51.372303 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366329 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:49:51.372303 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366333 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:49:51.372303 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366336 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:49:51.372303 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366338 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:49:51.372303 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366341 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:49:51.372303 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366343 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:49:51.372303 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366345 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:49:51.372303 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366348 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:49:51.372303 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366350 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:49:51.372303 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366352 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:49:51.372303 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366355 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:49:51.372303 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366357 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:49:51.372303 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366360 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:49:51.372303 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366362 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:49:51.372303 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366364 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:49:51.372660 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.366369 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 18:49:51.372660 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366469 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:49:51.372660 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366473 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:49:51.372660 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366476 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:49:51.372660 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366479 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:49:51.372660 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366482 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:49:51.372660 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366484 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:49:51.372660 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366487 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:49:51.372660 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366489 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:49:51.372660 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366492 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:49:51.372660 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366495 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:49:51.372660 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366497 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:49:51.372660 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366499 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:49:51.372660 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366502 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:49:51.372660 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366504 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:49:51.373017 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366506 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:49:51.373017 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366509 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:49:51.373017 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366511 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:49:51.373017 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366513 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:49:51.373017 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366516 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:49:51.373017 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366518 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:49:51.373017 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366520 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:49:51.373017 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366522 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:49:51.373017 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366525 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:49:51.373017 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366527 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:49:51.373017 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366529 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:49:51.373017 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366531 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:49:51.373017 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366534 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:49:51.373017 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366536 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:49:51.373017 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366539 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:49:51.373017 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366542 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:49:51.373017 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366544 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:49:51.373017 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366547 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:49:51.373017 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366549 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:49:51.373473 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366551 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:49:51.373473 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366553 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:49:51.373473 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366556 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:49:51.373473 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366558 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:49:51.373473 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366560 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:49:51.373473 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366563 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:49:51.373473 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366565 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:49:51.373473 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366567 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:49:51.373473 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366569 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:49:51.373473 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366572 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:49:51.373473 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366574 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:49:51.373473 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366576 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:49:51.373473 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366579 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:49:51.373473 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366581 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:49:51.373473 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366584 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:49:51.373473 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366586 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:49:51.373473 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366589 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:49:51.373473 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366591 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:49:51.373473 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366594 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:49:51.373913 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366598 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:49:51.373913 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366601 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:49:51.373913 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366603 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:49:51.373913 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366606 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:49:51.373913 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366608 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:49:51.373913 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366610 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:49:51.373913 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366613 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:49:51.373913 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366616 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:49:51.373913 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366619 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:49:51.373913 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366623 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:49:51.373913 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366626 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:49:51.373913 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366629 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:49:51.373913 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366631 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:49:51.373913 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366634 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:49:51.373913 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366636 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:49:51.373913 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366639 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:49:51.373913 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366641 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:49:51.373913 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366643 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:49:51.373913 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366646 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:49:51.374377 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366648 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:49:51.374377 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366650 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:49:51.374377 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366653 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:49:51.374377 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366655 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:49:51.374377 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366657 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:49:51.374377 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366660 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:49:51.374377 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366662 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:49:51.374377 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366664 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:49:51.374377 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366666 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:49:51.374377 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366669 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:49:51.374377 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366671 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:49:51.374377 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366673 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:49:51.374377 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366675 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:49:51.374377 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366677 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:49:51.374377 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366680 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:49:51.374377 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366682 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:49:51.374377 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366684 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:49:51.374377 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366686 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:49:51.374377 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366688 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:49:51.374377 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366691 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:49:51.374377 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366694 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:49:51.374872 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366696 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:49:51.374872 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366698 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:49:51.374872 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366701 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:49:51.374872 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366703 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:49:51.374872 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366706 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:49:51.374872 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366708 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:49:51.374872 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:51.366710 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:49:51.374872 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.366715 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 18:49:51.374872 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.366805 2564 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 18:49:51.374872 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.370101 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 18:49:51.374872 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.371969 2564 server.go:1019] "Starting client certificate rotation" Jul 07 18:49:51.374872 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.372073 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 18:49:51.374872 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.372109 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 18:49:51.375335 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.374671 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 18:49:51.377956 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.377943 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 18:49:51.384117 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.384100 2564 log.go:25] "Validated CRI v1 runtime API" Jul 07 18:49:51.386035 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.386018 2564 log.go:25] "Validated CRI v1 image API" Jul 07 18:49:51.386802 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.386787 2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 18:49:51.389740 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.389722 2564 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9e857cf0-f50a-4340-93ff-dea857743e30:/dev/nvme0n1p4 e622d534-1cee-4db1-beda-f1a1d88c1c13:/dev/nvme0n1p3] Jul 07 18:49:51.389796 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.389739 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 18:49:51.394481 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.394385 2564 manager.go:216] Machine: {Timestamp:2026-07-07 18:49:51.393079667 +0000 UTC m=+0.361968492 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3198932 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec225dd39d1a32d96228bbfb33df71ef SystemUUID:ec225dd3-9d1a-32d9-6228-bbfb33df71ef BootID:71ae89ac-4d8e-441d-b91f-59f23087b543 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:bc:dc:12:70:0d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:bc:dc:12:70:0d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:8e:c8:cd:ee:e6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 18:49:51.394481 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.394481 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 18:49:51.394565 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.394546 2564 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 18:49:51.398131 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.398104 2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 18:49:51.398248 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.398136 2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-66.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 18:49:51.399000 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.398989 2564 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 18:49:51.399033 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.399003 2564 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 18:49:51.399033 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.399010 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 18:49:51.399559 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.399549 2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 18:49:51.401016 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.401006 2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 18:49:51.401523 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.401509 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 18:49:51.404383 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.404373 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 07 18:49:51.404489 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.404480 2564 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 18:49:51.404523 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.404518 2564 kubelet.go:486] "Attempting to sync node with API server" Jul 07 18:49:51.404550 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.404531 2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 18:49:51.404550 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.404542 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 18:49:51.404550 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.404550 2564 kubelet.go:398] "Adding apiserver pod source" Jul 07 18:49:51.404632 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.404558 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 18:49:51.404874 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.404862 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 18:49:51.404916 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.404880 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 18:49:51.409185 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.409099 2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 18:49:51.411200 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.411182 2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 18:49:51.411264 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.411213 2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 18:49:51.412412 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.412401 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 18:49:51.412463 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.412417 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 18:49:51.412463 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.412422 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 18:49:51.412463 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.412428 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 18:49:51.412463 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.412436 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 18:49:51.412463 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.412441 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 18:49:51.412463 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.412448 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 18:49:51.412463 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.412453 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 18:49:51.412463 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.412459 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 18:49:51.412463 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.412464 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 18:49:51.412685 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.412472 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 18:49:51.412685 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.412477 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 18:49:51.414320 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.414303 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 18:49:51.414320 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.414314 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 18:49:51.415780 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:51.415762 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 18:49:51.415922 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:51.415904 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-66.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 18:49:51.416138 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.416122 2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-66.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 18:49:51.417498 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.417486 2564 server.go:1268] "Started kubelet" Jul 07 18:49:51.417631 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.417579 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 18:49:51.417720 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.417586 2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 18:49:51.417773 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.417745 2564 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 18:49:51.418245 ip-10-0-142-66 systemd[1]: Started Kubernetes Kubelet. Jul 07 18:49:51.419541 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.419519 2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 18:49:51.419744 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.419727 2564 server.go:310] "Adding debug handlers to kubelet server" Jul 07 18:49:51.424019 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:51.424001 2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 18:49:51.426178 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.426159 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 18:49:51.426255 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.426165 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 18:49:51.426313 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:51.426276 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-66.ec2.internal\" not found" Jul 07 18:49:51.426817 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.426803 2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 18:49:51.426920 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.426803 2564 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 18:49:51.426920 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.426919 2564 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 18:49:51.427084 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.426995 2564 factory.go:55] Registering systemd factory Jul 07 18:49:51.427084 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.427009 2564 factory.go:223] Registration of the systemd container factory successfully Jul 07 18:49:51.427084 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.427025 2564 reconstruct.go:97] "Volume reconstruction finished" Jul 07 18:49:51.427084 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.427035 2564 reconciler.go:29] "Reconciler: start to sync state" Jul 07 18:49:51.427285 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.427274 2564 factory.go:153] Registering CRI-O factory Jul 07 18:49:51.427315 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.427288 2564 factory.go:223] Registration of the crio container factory successfully Jul 07 18:49:51.427351 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.427329 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 18:49:51.427381 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.427358 2564 factory.go:103] Registering Raw factory Jul 07 18:49:51.427381 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.427374 2564 manager.go:1196] Started watching for new ooms in manager Jul 07 18:49:51.427855 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.427839 2564 manager.go:349] Starting recovery of all containers Jul 07 18:49:51.433121 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:51.433101 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 18:49:51.433203 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:51.433149 2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-66.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 18:49:51.434500 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:51.433168 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-66.ec2.internal.18c0167ffe0a3e76 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-66.ec2.internal,UID:ip-10-0-142-66.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-66.ec2.internal,},FirstTimestamp:2026-07-07 18:49:51.417458294 +0000 UTC m=+0.386347119,LastTimestamp:2026-07-07 18:49:51.417458294 +0000 UTC m=+0.386347119,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-66.ec2.internal,}" Jul 07 18:49:51.438152 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.437986 2564 manager.go:354] Recovery completed Jul 07 18:49:51.443849 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.443836 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:49:51.445585 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.445568 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:49:51.445635 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.445600 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:49:51.445635 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.445612 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:49:51.446199 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.446125 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 18:49:51.446199 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.446136 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 18:49:51.446199 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.446153 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 07 18:49:51.446355 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.446137 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:49:51.446829 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.446812 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:49:51.446890 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.446847 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:49:51.446890 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.446863 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:49:51.447466 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.447455 2564 policy_none.go:49] "None policy: Start" Jul 07 18:49:51.447502 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.447470 2564 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 18:49:51.447502 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.447479 2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 18:49:51.447890 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:51.447833 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-66.ec2.internal.18c0167fffb76e9f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-66.ec2.internal,UID:ip-10-0-142-66.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-66.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-66.ec2.internal,},FirstTimestamp:2026-07-07 18:49:51.445585567 +0000 UTC m=+0.414474395,LastTimestamp:2026-07-07 18:49:51.445585567 +0000 UTC m=+0.414474395,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-66.ec2.internal,}" Jul 07 18:49:51.448553 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.448542 2564 policy_none.go:47] "Start" Jul 07 18:49:51.456157 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:51.456101 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-66.ec2.internal.18c0167fffb7c3ce default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-66.ec2.internal,UID:ip-10-0-142-66.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-66.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-66.ec2.internal,},FirstTimestamp:2026-07-07 18:49:51.445607374 +0000 UTC m=+0.414496204,LastTimestamp:2026-07-07 18:49:51.445607374 +0000 UTC m=+0.414496204,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-66.ec2.internal,}" Jul 07 18:49:51.459173 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.459157 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vglwp" Jul 07 18:49:51.465850 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:51.465793 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-66.ec2.internal.18c0167fffb7e64f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-66.ec2.internal,UID:ip-10-0-142-66.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-142-66.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-142-66.ec2.internal,},FirstTimestamp:2026-07-07 18:49:51.445616207 +0000 UTC m=+0.414505035,LastTimestamp:2026-07-07 18:49:51.445616207 +0000 UTC m=+0.414505035,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-66.ec2.internal,}" Jul 07 18:49:51.470079 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.470060 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vglwp" Jul 07 18:49:51.488635 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.488621 2564 manager.go:337] "Starting Device Plugin manager" Jul 07 18:49:51.488719 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:51.488658 2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 18:49:51.488719 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.488671 2564 server.go:87] "Starting device plugin registration server" Jul 07 18:49:51.488903 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.488876 2564 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 18:49:51.489003 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.488892 2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 18:49:51.489003 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.488986 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 18:49:51.489136 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.489068 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 18:49:51.489136 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.489076 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 18:49:51.489548 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:51.489531 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 18:49:51.489610 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:51.489582 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-66.ec2.internal\" not found" Jul 07 18:49:51.545956 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.545927 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 18:49:51.547387 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.547167 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 18:49:51.547387 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.547198 2564 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 18:49:51.547387 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.547218 2564 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 18:49:51.547387 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:51.547263 2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 18:49:51.549772 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.549756 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 18:49:51.590017 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.589988 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:49:51.590753 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.590737 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:49:51.590833 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.590770 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:49:51.590833 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.590785 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:49:51.590833 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.590810 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-66.ec2.internal" Jul 07 18:49:51.603706 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.603665 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-66.ec2.internal" Jul 07 18:49:51.603706 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:51.603684 2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-66.ec2.internal\": node \"ip-10-0-142-66.ec2.internal\" not found" Jul 07 18:49:51.630326 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:51.630307 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-66.ec2.internal\" not found" Jul 07 18:49:51.647574 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.647555 2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-66.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-66.ec2.internal"] Jul 07 18:49:51.647625 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.647607 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:49:51.648318 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.648306 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:49:51.648378 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.648330 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:49:51.648378 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.648339 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:49:51.649506 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.649494 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:49:51.649621 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.649607 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-66.ec2.internal" Jul 07 18:49:51.649693 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.649632 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:49:51.650881 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.650290 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:49:51.650881 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.650292 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:49:51.650881 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.650340 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:49:51.650881 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.650341 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:49:51.650881 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.650355 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:49:51.650881 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.650363 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:49:51.651717 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.651701 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-66.ec2.internal" Jul 07 18:49:51.651774 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.651731 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:49:51.652416 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.652402 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:49:51.652518 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.652422 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:49:51.652518 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.652433 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:49:51.681576 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:51.681556 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-66.ec2.internal\" not found" node="ip-10-0-142-66.ec2.internal" Jul 07 18:49:51.685821 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:51.685809 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-66.ec2.internal\" not found" node="ip-10-0-142-66.ec2.internal" Jul 07 18:49:51.728817 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.728797 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/63225eaf1c1ca60865989b36d98a70bf-config\") pod \"kube-apiserver-proxy-ip-10-0-142-66.ec2.internal\" (UID: \"63225eaf1c1ca60865989b36d98a70bf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-66.ec2.internal" Jul 07 18:49:51.728895 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.728819 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b4755395f1b291c9d732ae38993723a3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-66.ec2.internal\" (UID: \"b4755395f1b291c9d732ae38993723a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-66.ec2.internal" Jul 07 18:49:51.728895 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.728835 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4755395f1b291c9d732ae38993723a3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-66.ec2.internal\" (UID: \"b4755395f1b291c9d732ae38993723a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-66.ec2.internal" Jul 07 18:49:51.730895 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:51.730882 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-66.ec2.internal\" not found" Jul 07 18:49:51.829498 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.829474 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4755395f1b291c9d732ae38993723a3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-66.ec2.internal\" (UID: \"b4755395f1b291c9d732ae38993723a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-66.ec2.internal" Jul 07 18:49:51.829598 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.829505 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/63225eaf1c1ca60865989b36d98a70bf-config\") pod \"kube-apiserver-proxy-ip-10-0-142-66.ec2.internal\" (UID: \"63225eaf1c1ca60865989b36d98a70bf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-66.ec2.internal" Jul 07 18:49:51.829598 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.829553 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b4755395f1b291c9d732ae38993723a3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-66.ec2.internal\" (UID: \"b4755395f1b291c9d732ae38993723a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-66.ec2.internal" Jul 07 18:49:51.829598 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.829565 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4755395f1b291c9d732ae38993723a3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-66.ec2.internal\" (UID: \"b4755395f1b291c9d732ae38993723a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-66.ec2.internal" Jul 07 18:49:51.829707 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.829607 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/63225eaf1c1ca60865989b36d98a70bf-config\") pod \"kube-apiserver-proxy-ip-10-0-142-66.ec2.internal\" (UID: \"63225eaf1c1ca60865989b36d98a70bf\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-66.ec2.internal" Jul 07 18:49:51.829707 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.829608 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b4755395f1b291c9d732ae38993723a3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-66.ec2.internal\" (UID: \"b4755395f1b291c9d732ae38993723a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-66.ec2.internal" Jul 07 18:49:51.831580 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:51.831568 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-66.ec2.internal\" not found" Jul 07 18:49:51.932260 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:51.932210 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-66.ec2.internal\" not found" Jul 07 18:49:51.985370 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.985356 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-66.ec2.internal" Jul 07 18:49:51.989417 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:51.989309 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-66.ec2.internal" Jul 07 18:49:52.033230 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:52.033211 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-66.ec2.internal\" not found" Jul 07 18:49:52.133705 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:52.133673 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-66.ec2.internal\" not found" Jul 07 18:49:52.234256 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:52.234211 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-66.ec2.internal\" not found" Jul 07 18:49:52.254009 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:52.253989 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 18:49:52.335025 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:52.335006 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-66.ec2.internal\" not found" Jul 07 18:49:52.371116 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:52.371100 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 18:49:52.371529 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:52.371197 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 18:49:52.371529 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:52.371243 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 18:49:52.435731 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:52.435707 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-66.ec2.internal\" not found" Jul 07 18:49:52.472525 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:52.472488 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 18:44:53 +0000 UTC" deadline="2028-03-27 00:46:32.066491728 +0000 UTC" Jul 07 18:49:52.472525 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:52.472524 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15077h56m39.593973614s" Jul 07 18:49:52.499312 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:52.499287 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63225eaf1c1ca60865989b36d98a70bf.slice/crio-0649cfcf7a47cb51cc3859311595c6b1fbaaa7c0d1423634b36e690acc7ad560 WatchSource:0}: Error finding container 0649cfcf7a47cb51cc3859311595c6b1fbaaa7c0d1423634b36e690acc7ad560: Status 404 returned error can't find the container with id 0649cfcf7a47cb51cc3859311595c6b1fbaaa7c0d1423634b36e690acc7ad560 Jul 07 18:49:52.499691 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:52.499672 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4755395f1b291c9d732ae38993723a3.slice/crio-9e270c36d54545ff2c9a7ca06fa92f3701ca1deab43dae7f13020d3ae86c7796 WatchSource:0}: Error finding container 9e270c36d54545ff2c9a7ca06fa92f3701ca1deab43dae7f13020d3ae86c7796: Status 404 returned error can't find the container with id 9e270c36d54545ff2c9a7ca06fa92f3701ca1deab43dae7f13020d3ae86c7796 Jul 07 18:49:52.505668 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:52.505644 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 18:49:52.536222 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:52.536201 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-66.ec2.internal\" not found" Jul 07 18:49:52.545887 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:52.545872 2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 18:49:52.550449 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:52.550404 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-66.ec2.internal" event={"ID":"b4755395f1b291c9d732ae38993723a3","Type":"ContainerStarted","Data":"9e270c36d54545ff2c9a7ca06fa92f3701ca1deab43dae7f13020d3ae86c7796"} Jul 07 18:49:52.551310 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:52.551286 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-66.ec2.internal" event={"ID":"63225eaf1c1ca60865989b36d98a70bf","Type":"ContainerStarted","Data":"0649cfcf7a47cb51cc3859311595c6b1fbaaa7c0d1423634b36e690acc7ad560"} Jul 07 18:49:52.626575 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:52.626561 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-66.ec2.internal" Jul 07 18:49:52.643196 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:52.643181 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 18:49:52.648683 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:52.648662 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-66.ec2.internal" Jul 07 18:49:52.665197 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:52.665179 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 18:49:54.436588 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.436559 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 18:49:54.956030 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.956001 2564 apiserver.go:52] "Watching apiserver" Jul 07 18:49:54.962670 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.962645 2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 18:49:54.964786 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.964758 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-cnvgc","openshift-dns/node-resolver-tmlg4","openshift-image-registry/node-ca-v6tvj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-66.ec2.internal","kube-system/global-pull-secret-syncer-hslvd","openshift-multus/multus-additional-cni-plugins-8pkcb","openshift-multus/multus-dg4c4","openshift-multus/network-metrics-daemon-hjnqt","openshift-network-diagnostics/network-check-target-4hhc9","openshift-network-operator/iptables-alerter-rq4nf","openshift-ovn-kubernetes/ovnkube-node-2tp24","kube-system/konnectivity-agent-vw7p6","kube-system/kube-apiserver-proxy-ip-10-0-142-66.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg"] Jul 07 18:49:54.966405 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.966385 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rq4nf" Jul 07 18:49:54.967792 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.967770 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:54.968713 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.968696 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 18:49:54.968805 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.968743 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fdngd\"" Jul 07 18:49:54.968805 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.968754 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 18:49:54.968904 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.968874 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:49:54.969033 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.969014 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v6tvj" Jul 07 18:49:54.969871 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.969849 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 18:49:54.969960 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.969888 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-sprrl\"" Jul 07 18:49:54.970486 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.970120 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:54.970486 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.970172 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 18:49:54.970486 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.970180 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 18:49:54.970486 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.970208 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 18:49:54.970486 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.970440 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 18:49:54.970852 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.970834 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 18:49:54.970966 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.970852 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 18:49:54.971055 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.971041 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 18:49:54.971680 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.971348 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 18:49:54.971680 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.971381 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 18:49:54.971680 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.971507 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:49:54.971680 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.971589 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-d86d4\"" Jul 07 18:49:54.971680 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:54.971565 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hslvd" podUID="2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf" Jul 07 18:49:54.972470 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.972444 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 18:49:54.972713 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.972698 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 18:49:54.972787 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.972752 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:54.972787 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.972772 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 18:49:54.973332 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.973316 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-p7l6n\"" Jul 07 18:49:54.974134 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.974115 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:49:54.974232 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:54.974192 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hhc9" podUID="6f83f082-dd3d-4223-b29b-f3c00bd060cd" Jul 07 18:49:54.975535 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.975516 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 18:49:54.975733 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.975717 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 18:49:54.975793 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.975749 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 18:49:54.975986 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.975969 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vdkg8\"" Jul 07 18:49:54.976132 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.976116 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 18:49:54.976221 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.976207 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 18:49:54.976276 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.976262 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 18:49:54.976326 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.976317 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:49:54.976393 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:54.976372 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjnqt" podUID="bcee6de1-03c6-4175-9e3b-c2f88c977d88" Jul 07 18:49:54.977145 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.976828 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tmlg4" Jul 07 18:49:54.978244 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.978215 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:54.979569 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.979374 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 18:49:54.979569 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.979441 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xvwfr\"" Jul 07 18:49:54.979569 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.979448 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 18:49:54.980321 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.980298 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dg4c4" Jul 07 18:49:54.981751 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.981431 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 18:49:54.981847 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.981816 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-s6rdr\"" Jul 07 18:49:54.982439 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.981901 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:49:54.983499 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.983479 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vw7p6" Jul 07 18:49:54.983768 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.983751 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 18:49:54.984130 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.984094 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lcbr5\"" Jul 07 18:49:54.985840 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.985824 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-24t8q\"" Jul 07 18:49:54.985959 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.985873 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 18:49:54.986056 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.985997 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 18:49:54.987650 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.987633 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-host-slash\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:54.987720 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.987656 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/52108ccf-1379-48c0-8477-5ab8914619dc-ovnkube-script-lib\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:54.987720 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.987671 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/09b26336-e7e1-40b3-b55c-f2640139c685-cnibin\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:54.987720 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.987685 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-etc-modprobe-d\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:54.987720 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.987704 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d21cfcc6-7fde-4272-86ff-26031eeacbda-device-dir\") pod \"aws-ebs-csi-driver-node-mh5jg\" (UID: \"d21cfcc6-7fde-4272-86ff-26031eeacbda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:54.987872 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.987726 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmd2h\" (UniqueName: \"kubernetes.io/projected/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-kube-api-access-nmd2h\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:54.987872 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.987751 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q86r2\" (UniqueName: \"kubernetes.io/projected/50f2f138-aebd-48b3-92eb-87c35347e4c6-kube-api-access-q86r2\") pod \"iptables-alerter-rq4nf\" (UID: \"50f2f138-aebd-48b3-92eb-87c35347e4c6\") " pod="openshift-network-operator/iptables-alerter-rq4nf" Jul 07 18:49:54.987872 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.987791 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-host-run-netns\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:54.987872 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.987816 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-run-systemd\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:54.987872 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.987838 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-sys\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:54.987872 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.987855 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66xml\" (UniqueName: \"kubernetes.io/projected/d21cfcc6-7fde-4272-86ff-26031eeacbda-kube-api-access-66xml\") pod \"aws-ebs-csi-driver-node-mh5jg\" (UID: \"d21cfcc6-7fde-4272-86ff-26031eeacbda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:54.988054 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.987884 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/09b26336-e7e1-40b3-b55c-f2640139c685-cni-binary-copy\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:54.988054 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.987904 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d305832c-a3ec-453e-9135-697b0b80c619-tmp-dir\") pod \"node-resolver-tmlg4\" (UID: \"d305832c-a3ec-453e-9135-697b0b80c619\") " pod="openshift-dns/node-resolver-tmlg4" Jul 07 18:49:54.988054 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.987918 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/52108ccf-1379-48c0-8477-5ab8914619dc-ovn-node-metrics-cert\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:54.988054 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.987932 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/09b26336-e7e1-40b3-b55c-f2640139c685-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:54.988054 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.987951 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret\") pod \"global-pull-secret-syncer-hslvd\" (UID: \"2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf\") " pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:49:54.988054 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.987975 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ac705cb6-fdca-4ffe-b46b-7e299b498b3e-serviceca\") pod \"node-ca-v6tvj\" (UID: \"ac705cb6-fdca-4ffe-b46b-7e299b498b3e\") " pod="openshift-image-registry/node-ca-v6tvj" Jul 07 18:49:54.988054 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988006 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gss2v\" (UniqueName: \"kubernetes.io/projected/09b26336-e7e1-40b3-b55c-f2640139c685-kube-api-access-gss2v\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:54.988054 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988024 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-tmp\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:54.988278 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988040 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-var-lib-openvswitch\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:54.988278 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988078 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-etc-openvswitch\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:54.988278 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988092 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:54.988278 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988107 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/52108ccf-1379-48c0-8477-5ab8914619dc-ovnkube-config\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:54.988278 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988146 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/09b26336-e7e1-40b3-b55c-f2640139c685-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:54.988278 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988175 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-etc-sysctl-conf\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:54.988278 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988198 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-host-kubelet\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:54.988278 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988219 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-run-ovn\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:54.988278 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988241 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-kubelet-config\") pod \"global-pull-secret-syncer-hslvd\" (UID: \"2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf\") " pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:49:54.988278 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988263 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09b26336-e7e1-40b3-b55c-f2640139c685-system-cni-dir\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:54.988278 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988279 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/09b26336-e7e1-40b3-b55c-f2640139c685-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:54.988677 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988301 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm6sr\" (UniqueName: \"kubernetes.io/projected/ac705cb6-fdca-4ffe-b46b-7e299b498b3e-kube-api-access-lm6sr\") pod \"node-ca-v6tvj\" (UID: \"ac705cb6-fdca-4ffe-b46b-7e299b498b3e\") " pod="openshift-image-registry/node-ca-v6tvj" Jul 07 18:49:54.988677 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988328 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d21cfcc6-7fde-4272-86ff-26031eeacbda-socket-dir\") pod \"aws-ebs-csi-driver-node-mh5jg\" (UID: \"d21cfcc6-7fde-4272-86ff-26031eeacbda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:54.988677 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988343 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d21cfcc6-7fde-4272-86ff-26031eeacbda-registration-dir\") pod \"aws-ebs-csi-driver-node-mh5jg\" (UID: \"d21cfcc6-7fde-4272-86ff-26031eeacbda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:54.988677 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988359 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-systemd-units\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:54.988677 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988376 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-run-openvswitch\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:54.988677 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988397 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-dbus\") pod \"global-pull-secret-syncer-hslvd\" (UID: \"2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf\") " pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:49:54.988677 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988416 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-etc-sysctl-d\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:54.988677 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988429 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d21cfcc6-7fde-4272-86ff-26031eeacbda-sys-fs\") pod \"aws-ebs-csi-driver-node-mh5jg\" (UID: \"d21cfcc6-7fde-4272-86ff-26031eeacbda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:54.988677 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988443 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/09b26336-e7e1-40b3-b55c-f2640139c685-os-release\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:54.988677 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988463 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-etc-systemd\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:54.988677 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988480 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d21cfcc6-7fde-4272-86ff-26031eeacbda-etc-selinux\") pod \"aws-ebs-csi-driver-node-mh5jg\" (UID: \"d21cfcc6-7fde-4272-86ff-26031eeacbda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:54.988677 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988496 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-host-cni-bin\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:54.988677 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988518 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-etc-sysconfig\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:54.988677 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988538 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-run\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:54.988677 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988555 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-node-log\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:54.988677 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988568 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-host-cni-netd\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:54.989222 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988611 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4kbb\" (UniqueName: \"kubernetes.io/projected/d305832c-a3ec-453e-9135-697b0b80c619-kube-api-access-r4kbb\") pod \"node-resolver-tmlg4\" (UID: \"d305832c-a3ec-453e-9135-697b0b80c619\") " pod="openshift-dns/node-resolver-tmlg4" Jul 07 18:49:54.989222 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988623 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-lib-modules\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:54.989222 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988635 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-etc-tuned\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:54.989222 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988656 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs\") pod \"network-metrics-daemon-hjnqt\" (UID: \"bcee6de1-03c6-4175-9e3b-c2f88c977d88\") " pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:49:54.989222 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988677 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/50f2f138-aebd-48b3-92eb-87c35347e4c6-iptables-alerter-script\") pod \"iptables-alerter-rq4nf\" (UID: \"50f2f138-aebd-48b3-92eb-87c35347e4c6\") " pod="openshift-network-operator/iptables-alerter-rq4nf" Jul 07 18:49:54.989222 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988691 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-log-socket\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:54.989222 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988706 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/52108ccf-1379-48c0-8477-5ab8914619dc-env-overrides\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:54.989222 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988731 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwbm9\" (UniqueName: \"kubernetes.io/projected/52108ccf-1379-48c0-8477-5ab8914619dc-kube-api-access-kwbm9\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:54.989222 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988756 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d21cfcc6-7fde-4272-86ff-26031eeacbda-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mh5jg\" (UID: \"d21cfcc6-7fde-4272-86ff-26031eeacbda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:54.989222 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988782 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d305832c-a3ec-453e-9135-697b0b80c619-hosts-file\") pod \"node-resolver-tmlg4\" (UID: \"d305832c-a3ec-453e-9135-697b0b80c619\") " pod="openshift-dns/node-resolver-tmlg4" Jul 07 18:49:54.989222 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988795 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-host\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:54.989222 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988828 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ac705cb6-fdca-4ffe-b46b-7e299b498b3e-host\") pod \"node-ca-v6tvj\" (UID: \"ac705cb6-fdca-4ffe-b46b-7e299b498b3e\") " pod="openshift-image-registry/node-ca-v6tvj" Jul 07 18:49:54.989222 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988860 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/50f2f138-aebd-48b3-92eb-87c35347e4c6-host-slash\") pod \"iptables-alerter-rq4nf\" (UID: \"50f2f138-aebd-48b3-92eb-87c35347e4c6\") " pod="openshift-network-operator/iptables-alerter-rq4nf" Jul 07 18:49:54.989222 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988885 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-host-run-ovn-kubernetes\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:54.989222 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988901 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-etc-kubernetes\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:54.989222 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988918 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-var-lib-kubelet\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:54.990009 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988943 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdhvc\" (UniqueName: \"kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc\") pod \"network-check-target-4hhc9\" (UID: \"6f83f082-dd3d-4223-b29b-f3c00bd060cd\") " pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:49:54.990009 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.988967 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz49f\" (UniqueName: \"kubernetes.io/projected/bcee6de1-03c6-4175-9e3b-c2f88c977d88-kube-api-access-wz49f\") pod \"network-metrics-daemon-hjnqt\" (UID: \"bcee6de1-03c6-4175-9e3b-c2f88c977d88\") " pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:49:54.993105 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:54.992030 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 18:49:55.008941 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.008924 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-k78hd" Jul 07 18:49:55.021649 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.021630 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-k78hd" Jul 07 18:49:55.077294 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.077266 2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 18:49:55.089155 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089131 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ac705cb6-fdca-4ffe-b46b-7e299b498b3e-host\") pod \"node-ca-v6tvj\" (UID: \"ac705cb6-fdca-4ffe-b46b-7e299b498b3e\") " pod="openshift-image-registry/node-ca-v6tvj" Jul 07 18:49:55.089155 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089156 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/50f2f138-aebd-48b3-92eb-87c35347e4c6-host-slash\") pod \"iptables-alerter-rq4nf\" (UID: \"50f2f138-aebd-48b3-92eb-87c35347e4c6\") " pod="openshift-network-operator/iptables-alerter-rq4nf" Jul 07 18:49:55.089303 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089172 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-host-run-ovn-kubernetes\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.089303 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089187 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-etc-kubernetes\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.089303 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089202 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-var-lib-kubelet\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.089303 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089244 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdhvc\" (UniqueName: \"kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc\") pod \"network-check-target-4hhc9\" (UID: \"6f83f082-dd3d-4223-b29b-f3c00bd060cd\") " pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:49:55.089303 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089255 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/50f2f138-aebd-48b3-92eb-87c35347e4c6-host-slash\") pod \"iptables-alerter-rq4nf\" (UID: \"50f2f138-aebd-48b3-92eb-87c35347e4c6\") " pod="openshift-network-operator/iptables-alerter-rq4nf" Jul 07 18:49:55.089303 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089267 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz49f\" (UniqueName: \"kubernetes.io/projected/bcee6de1-03c6-4175-9e3b-c2f88c977d88-kube-api-access-wz49f\") pod \"network-metrics-daemon-hjnqt\" (UID: \"bcee6de1-03c6-4175-9e3b-c2f88c977d88\") " pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:49:55.089303 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089257 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ac705cb6-fdca-4ffe-b46b-7e299b498b3e-host\") pod \"node-ca-v6tvj\" (UID: \"ac705cb6-fdca-4ffe-b46b-7e299b498b3e\") " pod="openshift-image-registry/node-ca-v6tvj" Jul 07 18:49:55.089303 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089281 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-host-run-ovn-kubernetes\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.089303 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089291 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-etc-kubernetes\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.089303 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089293 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-host-var-lib-kubelet\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.089741 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089309 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-var-lib-kubelet\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.089741 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089340 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-host-slash\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.089741 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089368 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/52108ccf-1379-48c0-8477-5ab8914619dc-ovnkube-script-lib\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.089741 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089396 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/09b26336-e7e1-40b3-b55c-f2640139c685-cnibin\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:55.089741 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089427 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-host-slash\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.089741 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089439 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/09b26336-e7e1-40b3-b55c-f2640139c685-cnibin\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:55.089741 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089478 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-etc-modprobe-d\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.089741 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089508 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d21cfcc6-7fde-4272-86ff-26031eeacbda-device-dir\") pod \"aws-ebs-csi-driver-node-mh5jg\" (UID: \"d21cfcc6-7fde-4272-86ff-26031eeacbda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:55.089741 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089535 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b1d9d37-d1f8-4ebd-b96d-0a11e50a90e4-agent-certs\") pod \"konnectivity-agent-vw7p6\" (UID: \"4b1d9d37-d1f8-4ebd-b96d-0a11e50a90e4\") " pod="kube-system/konnectivity-agent-vw7p6" Jul 07 18:49:55.089741 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089564 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d21cfcc6-7fde-4272-86ff-26031eeacbda-device-dir\") pod \"aws-ebs-csi-driver-node-mh5jg\" (UID: \"d21cfcc6-7fde-4272-86ff-26031eeacbda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:55.089741 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089590 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmd2h\" (UniqueName: \"kubernetes.io/projected/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-kube-api-access-nmd2h\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.089741 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089623 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q86r2\" (UniqueName: \"kubernetes.io/projected/50f2f138-aebd-48b3-92eb-87c35347e4c6-kube-api-access-q86r2\") pod \"iptables-alerter-rq4nf\" (UID: \"50f2f138-aebd-48b3-92eb-87c35347e4c6\") " pod="openshift-network-operator/iptables-alerter-rq4nf" Jul 07 18:49:55.089741 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089633 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-etc-modprobe-d\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.089741 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089649 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-host-run-netns\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.089741 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089672 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-run-systemd\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.089741 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089695 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-sys\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.089741 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089714 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-host-run-netns\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.090556 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089720 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66xml\" (UniqueName: \"kubernetes.io/projected/d21cfcc6-7fde-4272-86ff-26031eeacbda-kube-api-access-66xml\") pod \"aws-ebs-csi-driver-node-mh5jg\" (UID: \"d21cfcc6-7fde-4272-86ff-26031eeacbda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:55.090556 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089756 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-host-run-k8s-cni-cncf-io\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.090556 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089767 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-run-systemd\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.090556 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089781 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-hostroot\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.090556 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089804 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-sys\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.090556 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089818 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/09b26336-e7e1-40b3-b55c-f2640139c685-cni-binary-copy\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:55.090556 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089837 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d305832c-a3ec-453e-9135-697b0b80c619-tmp-dir\") pod \"node-resolver-tmlg4\" (UID: \"d305832c-a3ec-453e-9135-697b0b80c619\") " pod="openshift-dns/node-resolver-tmlg4" Jul 07 18:49:55.090556 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089852 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-multus-conf-dir\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.090556 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089871 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/52108ccf-1379-48c0-8477-5ab8914619dc-ovn-node-metrics-cert\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.090556 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089881 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/52108ccf-1379-48c0-8477-5ab8914619dc-ovnkube-script-lib\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.090556 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089897 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/09b26336-e7e1-40b3-b55c-f2640139c685-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:55.090556 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089925 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-cnibin\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.090556 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089949 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-host-var-lib-cni-multus\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.090556 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089973 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret\") pod \"global-pull-secret-syncer-hslvd\" (UID: \"2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf\") " pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:49:55.090556 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.089996 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ac705cb6-fdca-4ffe-b46b-7e299b498b3e-serviceca\") pod \"node-ca-v6tvj\" (UID: \"ac705cb6-fdca-4ffe-b46b-7e299b498b3e\") " pod="openshift-image-registry/node-ca-v6tvj" Jul 07 18:49:55.090556 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090016 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-cni-binary-copy\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.090556 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090036 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-etc-kubernetes\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.091298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090067 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf82d\" (UniqueName: \"kubernetes.io/projected/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-kube-api-access-bf82d\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.091298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090089 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gss2v\" (UniqueName: \"kubernetes.io/projected/09b26336-e7e1-40b3-b55c-f2640139c685-kube-api-access-gss2v\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:55.091298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090109 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-multus-socket-dir-parent\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.091298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090133 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b1d9d37-d1f8-4ebd-b96d-0a11e50a90e4-konnectivity-ca\") pod \"konnectivity-agent-vw7p6\" (UID: \"4b1d9d37-d1f8-4ebd-b96d-0a11e50a90e4\") " pod="kube-system/konnectivity-agent-vw7p6" Jul 07 18:49:55.091298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090154 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-tmp\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.091298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090185 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-var-lib-openvswitch\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.091298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090208 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-etc-openvswitch\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.091298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090219 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 18:49:55.091298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090233 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.091298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090260 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/52108ccf-1379-48c0-8477-5ab8914619dc-ovnkube-config\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.091298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090284 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/09b26336-e7e1-40b3-b55c-f2640139c685-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:55.091298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090307 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-etc-sysctl-conf\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.091298 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:55.090329 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:49:55.091298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090352 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-host-kubelet\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.091298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090377 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-run-ovn\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.091298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090400 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-kubelet-config\") pod \"global-pull-secret-syncer-hslvd\" (UID: \"2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf\") " pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:49:55.091298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090436 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/09b26336-e7e1-40b3-b55c-f2640139c685-cni-binary-copy\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:55.091298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090472 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/09b26336-e7e1-40b3-b55c-f2640139c685-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:55.092137 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090486 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.092137 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:55.090556 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret podName:2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf nodeName:}" failed. No retries permitted until 2026-07-07 18:49:55.590532609 +0000 UTC m=+3.009892811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret") pod "global-pull-secret-syncer-hslvd" (UID: "2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:49:55.092137 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090220 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d305832c-a3ec-453e-9135-697b0b80c619-tmp-dir\") pod \"node-resolver-tmlg4\" (UID: \"d305832c-a3ec-453e-9135-697b0b80c619\") " pod="openshift-dns/node-resolver-tmlg4" Jul 07 18:49:55.092137 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090653 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-kubelet-config\") pod \"global-pull-secret-syncer-hslvd\" (UID: \"2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf\") " pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:49:55.092137 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090658 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-var-lib-openvswitch\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.092137 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090694 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-host-kubelet\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.092137 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090644 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09b26336-e7e1-40b3-b55c-f2640139c685-system-cni-dir\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:55.092137 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090715 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-run-ovn\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.092137 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090756 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-etc-openvswitch\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.092137 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090892 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/09b26336-e7e1-40b3-b55c-f2640139c685-system-cni-dir\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:55.092137 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090918 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/09b26336-e7e1-40b3-b55c-f2640139c685-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:55.092137 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090941 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ac705cb6-fdca-4ffe-b46b-7e299b498b3e-serviceca\") pod \"node-ca-v6tvj\" (UID: \"ac705cb6-fdca-4ffe-b46b-7e299b498b3e\") " pod="openshift-image-registry/node-ca-v6tvj" Jul 07 18:49:55.092137 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090944 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm6sr\" (UniqueName: \"kubernetes.io/projected/ac705cb6-fdca-4ffe-b46b-7e299b498b3e-kube-api-access-lm6sr\") pod \"node-ca-v6tvj\" (UID: \"ac705cb6-fdca-4ffe-b46b-7e299b498b3e\") " pod="openshift-image-registry/node-ca-v6tvj" Jul 07 18:49:55.092137 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.090991 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d21cfcc6-7fde-4272-86ff-26031eeacbda-socket-dir\") pod \"aws-ebs-csi-driver-node-mh5jg\" (UID: \"d21cfcc6-7fde-4272-86ff-26031eeacbda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:55.092137 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091015 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d21cfcc6-7fde-4272-86ff-26031eeacbda-registration-dir\") pod \"aws-ebs-csi-driver-node-mh5jg\" (UID: \"d21cfcc6-7fde-4272-86ff-26031eeacbda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:55.092137 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091038 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-systemd-units\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.093046 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091060 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-run-openvswitch\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.093046 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091072 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/09b26336-e7e1-40b3-b55c-f2640139c685-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:55.093046 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091082 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-dbus\") pod \"global-pull-secret-syncer-hslvd\" (UID: \"2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf\") " pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:49:55.093046 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091107 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-etc-sysctl-d\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.093046 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091132 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d21cfcc6-7fde-4272-86ff-26031eeacbda-sys-fs\") pod \"aws-ebs-csi-driver-node-mh5jg\" (UID: \"d21cfcc6-7fde-4272-86ff-26031eeacbda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:55.093046 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091159 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-os-release\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.093046 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091173 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/52108ccf-1379-48c0-8477-5ab8914619dc-ovnkube-config\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.093046 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091193 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/09b26336-e7e1-40b3-b55c-f2640139c685-os-release\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:55.093046 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091178 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d21cfcc6-7fde-4272-86ff-26031eeacbda-socket-dir\") pod \"aws-ebs-csi-driver-node-mh5jg\" (UID: \"d21cfcc6-7fde-4272-86ff-26031eeacbda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:55.093046 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091208 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/09b26336-e7e1-40b3-b55c-f2640139c685-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:55.093046 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091216 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-etc-systemd\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.093046 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091240 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-systemd-units\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.093046 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091207 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-dbus\") pod \"global-pull-secret-syncer-hslvd\" (UID: \"2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf\") " pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:49:55.093046 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091282 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d21cfcc6-7fde-4272-86ff-26031eeacbda-registration-dir\") pod \"aws-ebs-csi-driver-node-mh5jg\" (UID: \"d21cfcc6-7fde-4272-86ff-26031eeacbda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:55.093046 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091270 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d21cfcc6-7fde-4272-86ff-26031eeacbda-sys-fs\") pod \"aws-ebs-csi-driver-node-mh5jg\" (UID: \"d21cfcc6-7fde-4272-86ff-26031eeacbda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:55.093046 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091299 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-etc-sysctl-d\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.093046 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091323 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-etc-sysctl-conf\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.093863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091336 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-etc-systemd\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.093863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091341 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-run-openvswitch\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.093863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091361 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d21cfcc6-7fde-4272-86ff-26031eeacbda-etc-selinux\") pod \"aws-ebs-csi-driver-node-mh5jg\" (UID: \"d21cfcc6-7fde-4272-86ff-26031eeacbda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:55.093863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091376 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/09b26336-e7e1-40b3-b55c-f2640139c685-os-release\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:55.093863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091395 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-system-cni-dir\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.093863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091419 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-host-run-netns\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.093863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091444 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-host-run-multus-certs\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.093863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091470 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-host-cni-bin\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.093863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091483 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d21cfcc6-7fde-4272-86ff-26031eeacbda-etc-selinux\") pod \"aws-ebs-csi-driver-node-mh5jg\" (UID: \"d21cfcc6-7fde-4272-86ff-26031eeacbda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:55.093863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091496 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-etc-sysconfig\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.093863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091520 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-run\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.093863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091541 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-multus-cni-dir\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.093863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091564 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-host-var-lib-cni-bin\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.093863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091592 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-etc-sysconfig\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.093863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091607 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-multus-daemon-config\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.093863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091630 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-run\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.093863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091632 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-node-log\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.094700 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091667 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-host-cni-netd\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.094700 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091672 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-node-log\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.094700 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091700 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4kbb\" (UniqueName: \"kubernetes.io/projected/d305832c-a3ec-453e-9135-697b0b80c619-kube-api-access-r4kbb\") pod \"node-resolver-tmlg4\" (UID: \"d305832c-a3ec-453e-9135-697b0b80c619\") " pod="openshift-dns/node-resolver-tmlg4" Jul 07 18:49:55.094700 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091714 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-host-cni-netd\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.094700 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091725 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-lib-modules\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.094700 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091736 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-host-cni-bin\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.094700 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091825 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-lib-modules\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.094700 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091834 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-etc-tuned\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.094700 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091863 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs\") pod \"network-metrics-daemon-hjnqt\" (UID: \"bcee6de1-03c6-4175-9e3b-c2f88c977d88\") " pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:49:55.094700 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091887 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/50f2f138-aebd-48b3-92eb-87c35347e4c6-iptables-alerter-script\") pod \"iptables-alerter-rq4nf\" (UID: \"50f2f138-aebd-48b3-92eb-87c35347e4c6\") " pod="openshift-network-operator/iptables-alerter-rq4nf" Jul 07 18:49:55.094700 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091910 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-log-socket\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.094700 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091930 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/52108ccf-1379-48c0-8477-5ab8914619dc-env-overrides\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.094700 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091957 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwbm9\" (UniqueName: \"kubernetes.io/projected/52108ccf-1379-48c0-8477-5ab8914619dc-kube-api-access-kwbm9\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.094700 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:55.091967 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:49:55.094700 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.091979 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d21cfcc6-7fde-4272-86ff-26031eeacbda-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mh5jg\" (UID: \"d21cfcc6-7fde-4272-86ff-26031eeacbda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:55.094700 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.092002 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d305832c-a3ec-453e-9135-697b0b80c619-hosts-file\") pod \"node-resolver-tmlg4\" (UID: \"d305832c-a3ec-453e-9135-697b0b80c619\") " pod="openshift-dns/node-resolver-tmlg4" Jul 07 18:49:55.094700 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:55.092038 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs podName:bcee6de1-03c6-4175-9e3b-c2f88c977d88 nodeName:}" failed. No retries permitted until 2026-07-07 18:49:55.592001781 +0000 UTC m=+3.011361997 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs") pod "network-metrics-daemon-hjnqt" (UID: "bcee6de1-03c6-4175-9e3b-c2f88c977d88") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:49:55.095235 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.092048 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d305832c-a3ec-453e-9135-697b0b80c619-hosts-file\") pod \"node-resolver-tmlg4\" (UID: \"d305832c-a3ec-453e-9135-697b0b80c619\") " pod="openshift-dns/node-resolver-tmlg4" Jul 07 18:49:55.095235 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.092066 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-host\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.095235 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.092146 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-host\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.095235 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.092198 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/52108ccf-1379-48c0-8477-5ab8914619dc-log-socket\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.095235 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.092243 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d21cfcc6-7fde-4272-86ff-26031eeacbda-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mh5jg\" (UID: \"d21cfcc6-7fde-4272-86ff-26031eeacbda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:55.095235 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.092374 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/52108ccf-1379-48c0-8477-5ab8914619dc-env-overrides\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.095235 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.092408 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/50f2f138-aebd-48b3-92eb-87c35347e4c6-iptables-alerter-script\") pod \"iptables-alerter-rq4nf\" (UID: \"50f2f138-aebd-48b3-92eb-87c35347e4c6\") " pod="openshift-network-operator/iptables-alerter-rq4nf" Jul 07 18:49:55.095235 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.094115 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-tmp\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.095235 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.094188 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/52108ccf-1379-48c0-8477-5ab8914619dc-ovn-node-metrics-cert\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.095235 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.094771 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-etc-tuned\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.101170 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:55.101150 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:49:55.101284 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:55.101177 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:49:55.101284 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:55.101197 2564 projected.go:196] Error preparing data for projected volume kube-api-access-sdhvc for pod openshift-network-diagnostics/network-check-target-4hhc9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:49:55.101284 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:55.101259 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc podName:6f83f082-dd3d-4223-b29b-f3c00bd060cd nodeName:}" failed. No retries permitted until 2026-07-07 18:49:55.601237253 +0000 UTC m=+3.020597480 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-sdhvc" (UniqueName: "kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc") pod "network-check-target-4hhc9" (UID: "6f83f082-dd3d-4223-b29b-f3c00bd060cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:49:55.102902 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.102172 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz49f\" (UniqueName: \"kubernetes.io/projected/bcee6de1-03c6-4175-9e3b-c2f88c977d88-kube-api-access-wz49f\") pod \"network-metrics-daemon-hjnqt\" (UID: \"bcee6de1-03c6-4175-9e3b-c2f88c977d88\") " pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:49:55.102902 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.102457 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q86r2\" (UniqueName: \"kubernetes.io/projected/50f2f138-aebd-48b3-92eb-87c35347e4c6-kube-api-access-q86r2\") pod \"iptables-alerter-rq4nf\" (UID: \"50f2f138-aebd-48b3-92eb-87c35347e4c6\") " pod="openshift-network-operator/iptables-alerter-rq4nf" Jul 07 18:49:55.106976 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.106909 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwbm9\" (UniqueName: \"kubernetes.io/projected/52108ccf-1379-48c0-8477-5ab8914619dc-kube-api-access-kwbm9\") pod \"ovnkube-node-2tp24\" (UID: \"52108ccf-1379-48c0-8477-5ab8914619dc\") " pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.107075 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.106976 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gss2v\" (UniqueName: \"kubernetes.io/projected/09b26336-e7e1-40b3-b55c-f2640139c685-kube-api-access-gss2v\") pod \"multus-additional-cni-plugins-8pkcb\" (UID: \"09b26336-e7e1-40b3-b55c-f2640139c685\") " pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:55.107866 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.107841 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmd2h\" (UniqueName: \"kubernetes.io/projected/28c0d463-ad7d-4ae5-b801-1dbe90cb7c55-kube-api-access-nmd2h\") pod \"tuned-cnvgc\" (UID: \"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55\") " pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.108166 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.108144 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66xml\" (UniqueName: \"kubernetes.io/projected/d21cfcc6-7fde-4272-86ff-26031eeacbda-kube-api-access-66xml\") pod \"aws-ebs-csi-driver-node-mh5jg\" (UID: \"d21cfcc6-7fde-4272-86ff-26031eeacbda\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:55.108262 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.108192 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm6sr\" (UniqueName: \"kubernetes.io/projected/ac705cb6-fdca-4ffe-b46b-7e299b498b3e-kube-api-access-lm6sr\") pod \"node-ca-v6tvj\" (UID: \"ac705cb6-fdca-4ffe-b46b-7e299b498b3e\") " pod="openshift-image-registry/node-ca-v6tvj" Jul 07 18:49:55.109155 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.109137 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4kbb\" (UniqueName: \"kubernetes.io/projected/d305832c-a3ec-453e-9135-697b0b80c619-kube-api-access-r4kbb\") pod \"node-resolver-tmlg4\" (UID: \"d305832c-a3ec-453e-9135-697b0b80c619\") " pod="openshift-dns/node-resolver-tmlg4" Jul 07 18:49:55.192718 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.192683 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-os-release\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.192860 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.192724 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-system-cni-dir\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.192860 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.192742 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-host-run-netns\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.192860 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.192765 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-host-run-multus-certs\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.192860 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.192787 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-multus-cni-dir\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.192860 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.192808 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-host-var-lib-cni-bin\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.192860 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.192827 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-os-release\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.192860 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.192837 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-system-cni-dir\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.192860 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.192837 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-host-run-netns\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.193260 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.192829 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-multus-daemon-config\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.193260 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.192878 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-host-run-multus-certs\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.193260 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.192899 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-multus-cni-dir\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.193260 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.192914 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-host-var-lib-cni-bin\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.193260 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.192935 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-host-var-lib-kubelet\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.193260 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.192961 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b1d9d37-d1f8-4ebd-b96d-0a11e50a90e4-agent-certs\") pod \"konnectivity-agent-vw7p6\" (UID: \"4b1d9d37-d1f8-4ebd-b96d-0a11e50a90e4\") " pod="kube-system/konnectivity-agent-vw7p6" Jul 07 18:49:55.193260 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.192989 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-host-run-k8s-cni-cncf-io\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.193260 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.193030 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-host-run-k8s-cni-cncf-io\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.193260 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.193052 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-hostroot\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.193260 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.193071 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-multus-conf-dir\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.193260 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.193088 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-cnibin\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.193260 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.193101 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-host-var-lib-cni-multus\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.193260 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.193135 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-cni-binary-copy\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.193260 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.193157 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-etc-kubernetes\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.193260 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.193181 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf82d\" (UniqueName: \"kubernetes.io/projected/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-kube-api-access-bf82d\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.193260 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.193205 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-multus-socket-dir-parent\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.193260 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.193222 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b1d9d37-d1f8-4ebd-b96d-0a11e50a90e4-konnectivity-ca\") pod \"konnectivity-agent-vw7p6\" (UID: \"4b1d9d37-d1f8-4ebd-b96d-0a11e50a90e4\") " pod="kube-system/konnectivity-agent-vw7p6" Jul 07 18:49:55.194025 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.193307 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-host-var-lib-cni-multus\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.194025 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.193347 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-multus-daemon-config\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.194025 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.193356 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-hostroot\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.194025 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.193383 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-multus-conf-dir\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.194025 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.193423 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-etc-kubernetes\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.194025 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.193429 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-cnibin\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.194025 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.193633 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-host-var-lib-kubelet\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.194025 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.193674 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-multus-socket-dir-parent\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.194025 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.193690 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b1d9d37-d1f8-4ebd-b96d-0a11e50a90e4-konnectivity-ca\") pod \"konnectivity-agent-vw7p6\" (UID: \"4b1d9d37-d1f8-4ebd-b96d-0a11e50a90e4\") " pod="kube-system/konnectivity-agent-vw7p6" Jul 07 18:49:55.194025 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.193881 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-cni-binary-copy\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.195652 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.195627 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b1d9d37-d1f8-4ebd-b96d-0a11e50a90e4-agent-certs\") pod \"konnectivity-agent-vw7p6\" (UID: \"4b1d9d37-d1f8-4ebd-b96d-0a11e50a90e4\") " pod="kube-system/konnectivity-agent-vw7p6" Jul 07 18:49:55.202274 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.202258 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf82d\" (UniqueName: \"kubernetes.io/projected/217e2ad4-6057-41e0-a4e8-3d2bd461fd27-kube-api-access-bf82d\") pod \"multus-dg4c4\" (UID: \"217e2ad4-6057-41e0-a4e8-3d2bd461fd27\") " pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.280056 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.279999 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rq4nf" Jul 07 18:49:55.287773 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.287755 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:49:55.300481 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.300461 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dg4c4" Jul 07 18:49:55.306008 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.305985 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v6tvj" Jul 07 18:49:55.312556 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.312533 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" Jul 07 18:49:55.320222 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.320201 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8pkcb" Jul 07 18:49:55.326505 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.326488 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tmlg4" Jul 07 18:49:55.333784 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.333770 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" Jul 07 18:49:55.338215 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.338200 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vw7p6" Jul 07 18:49:55.595819 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.595745 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret\") pod \"global-pull-secret-syncer-hslvd\" (UID: \"2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf\") " pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:49:55.595819 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.595790 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs\") pod \"network-metrics-daemon-hjnqt\" (UID: \"bcee6de1-03c6-4175-9e3b-c2f88c977d88\") " pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:49:55.595974 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:55.595892 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:49:55.595974 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:55.595953 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret podName:2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf nodeName:}" failed. No retries permitted until 2026-07-07 18:49:56.595939425 +0000 UTC m=+4.015299628 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret") pod "global-pull-secret-syncer-hslvd" (UID: "2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:49:55.596041 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:55.595893 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:49:55.596041 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:55.596022 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs podName:bcee6de1-03c6-4175-9e3b-c2f88c977d88 nodeName:}" failed. No retries permitted until 2026-07-07 18:49:56.596010211 +0000 UTC m=+4.015370413 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs") pod "network-metrics-daemon-hjnqt" (UID: "bcee6de1-03c6-4175-9e3b-c2f88c977d88") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:49:55.696892 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:55.696859 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdhvc\" (UniqueName: \"kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc\") pod \"network-check-target-4hhc9\" (UID: \"6f83f082-dd3d-4223-b29b-f3c00bd060cd\") " pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:49:55.697060 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:55.696990 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:49:55.697060 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:55.697014 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:49:55.697060 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:55.697026 2564 projected.go:196] Error preparing data for projected volume kube-api-access-sdhvc for pod openshift-network-diagnostics/network-check-target-4hhc9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:49:55.697178 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:55.697086 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc podName:6f83f082-dd3d-4223-b29b-f3c00bd060cd nodeName:}" failed. No retries permitted until 2026-07-07 18:49:56.697067834 +0000 UTC m=+4.116428045 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-sdhvc" (UniqueName: "kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc") pod "network-check-target-4hhc9" (UID: "6f83f082-dd3d-4223-b29b-f3c00bd060cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:49:55.913763 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:55.913732 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b1d9d37_d1f8_4ebd_b96d_0a11e50a90e4.slice/crio-eb3ef6fb5c4f9da3435c31cff160e4f747c8a57ea31db9bc405b06f28f88e613 WatchSource:0}: Error finding container eb3ef6fb5c4f9da3435c31cff160e4f747c8a57ea31db9bc405b06f28f88e613: Status 404 returned error can't find the container with id eb3ef6fb5c4f9da3435c31cff160e4f747c8a57ea31db9bc405b06f28f88e613 Jul 07 18:49:55.914940 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:55.914788 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac705cb6_fdca_4ffe_b46b_7e299b498b3e.slice/crio-9feead14f826857903cf5b153ffec86a9ff91c48f2819db69598398cb295c2a6 WatchSource:0}: Error finding container 9feead14f826857903cf5b153ffec86a9ff91c48f2819db69598398cb295c2a6: Status 404 returned error can't find the container with id 9feead14f826857903cf5b153ffec86a9ff91c48f2819db69598398cb295c2a6 Jul 07 18:49:55.917918 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:55.917893 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd21cfcc6_7fde_4272_86ff_26031eeacbda.slice/crio-92c14999ba4349468ad422bfa9ce8316726afc72a090bb86c72ae96e23d819b9 WatchSource:0}: Error finding container 92c14999ba4349468ad422bfa9ce8316726afc72a090bb86c72ae96e23d819b9: Status 404 returned error can't find the container with id 92c14999ba4349468ad422bfa9ce8316726afc72a090bb86c72ae96e23d819b9 Jul 07 18:49:55.918607 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:55.918443 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09b26336_e7e1_40b3_b55c_f2640139c685.slice/crio-c94535072d11d2909d82ac112be5e1a2e2782391fcf654a6d24b0fd4e28190fc WatchSource:0}: Error finding container c94535072d11d2909d82ac112be5e1a2e2782391fcf654a6d24b0fd4e28190fc: Status 404 returned error can't find the container with id c94535072d11d2909d82ac112be5e1a2e2782391fcf654a6d24b0fd4e28190fc Jul 07 18:49:55.919269 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:55.919245 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c0d463_ad7d_4ae5_b801_1dbe90cb7c55.slice/crio-202aec592fd367410d188772f80a9fac9afe2d3e252529df724d2bdbd23a028e WatchSource:0}: Error finding container 202aec592fd367410d188772f80a9fac9afe2d3e252529df724d2bdbd23a028e: Status 404 returned error can't find the container with id 202aec592fd367410d188772f80a9fac9afe2d3e252529df724d2bdbd23a028e Jul 07 18:49:55.920959 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:55.920935 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50f2f138_aebd_48b3_92eb_87c35347e4c6.slice/crio-b2ff7ee3bde6d90d9d736aee13c1bf6d45b7e9eff9171cbd9057622af18352ec WatchSource:0}: Error finding container b2ff7ee3bde6d90d9d736aee13c1bf6d45b7e9eff9171cbd9057622af18352ec: Status 404 returned error can't find the container with id b2ff7ee3bde6d90d9d736aee13c1bf6d45b7e9eff9171cbd9057622af18352ec Jul 07 18:49:55.921802 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:55.921781 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd305832c_a3ec_453e_9135_697b0b80c619.slice/crio-1fdd65a7de18e72b571eda641f46e07cd057039bf6eecd3c98ed62d6430bfd67 WatchSource:0}: Error finding container 1fdd65a7de18e72b571eda641f46e07cd057039bf6eecd3c98ed62d6430bfd67: Status 404 returned error can't find the container with id 1fdd65a7de18e72b571eda641f46e07cd057039bf6eecd3c98ed62d6430bfd67 Jul 07 18:49:55.923652 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:49:55.923628 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52108ccf_1379_48c0_8477_5ab8914619dc.slice/crio-4199ea81f4da46fe0c1bddf9ecd30b8fc9c786c21fb0baf8b82396ec713b4c0c WatchSource:0}: Error finding container 4199ea81f4da46fe0c1bddf9ecd30b8fc9c786c21fb0baf8b82396ec713b4c0c: Status 404 returned error can't find the container with id 4199ea81f4da46fe0c1bddf9ecd30b8fc9c786c21fb0baf8b82396ec713b4c0c Jul 07 18:49:56.023732 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:56.023517 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 18:44:55 +0000 UTC" deadline="2028-02-23 18:49:21.928171372 +0000 UTC" Jul 07 18:49:56.023732 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:56.023727 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14303h59m25.904451082s" Jul 07 18:49:56.097335 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:56.097308 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:49:56.097483 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:56.097308 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:49:56.097483 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:56.097431 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hslvd" podUID="2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf" Jul 07 18:49:56.097483 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:56.097471 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hhc9" podUID="6f83f082-dd3d-4223-b29b-f3c00bd060cd" Jul 07 18:49:56.106236 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:56.106208 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dg4c4" event={"ID":"217e2ad4-6057-41e0-a4e8-3d2bd461fd27","Type":"ContainerStarted","Data":"6432b1ca64120895d4dbe5f68a6bccd41b9ca721cb7a3c0173d4f4b2b34b493b"} Jul 07 18:49:56.107309 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:56.107277 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rq4nf" event={"ID":"50f2f138-aebd-48b3-92eb-87c35347e4c6","Type":"ContainerStarted","Data":"b2ff7ee3bde6d90d9d736aee13c1bf6d45b7e9eff9171cbd9057622af18352ec"} Jul 07 18:49:56.108479 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:56.108453 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8pkcb" event={"ID":"09b26336-e7e1-40b3-b55c-f2640139c685","Type":"ContainerStarted","Data":"c94535072d11d2909d82ac112be5e1a2e2782391fcf654a6d24b0fd4e28190fc"} Jul 07 18:49:56.109517 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:56.109497 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vw7p6" event={"ID":"4b1d9d37-d1f8-4ebd-b96d-0a11e50a90e4","Type":"ContainerStarted","Data":"eb3ef6fb5c4f9da3435c31cff160e4f747c8a57ea31db9bc405b06f28f88e613"} Jul 07 18:49:56.111241 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:56.111221 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-66.ec2.internal" event={"ID":"63225eaf1c1ca60865989b36d98a70bf","Type":"ContainerStarted","Data":"ac07d1b35b64e71540c9836b0371ba349d747795b420baae6b7e6f124b74e339"} Jul 07 18:49:56.112404 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:56.112373 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" event={"ID":"d21cfcc6-7fde-4272-86ff-26031eeacbda","Type":"ContainerStarted","Data":"92c14999ba4349468ad422bfa9ce8316726afc72a090bb86c72ae96e23d819b9"} Jul 07 18:49:56.113461 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:56.113439 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" event={"ID":"52108ccf-1379-48c0-8477-5ab8914619dc","Type":"ContainerStarted","Data":"4199ea81f4da46fe0c1bddf9ecd30b8fc9c786c21fb0baf8b82396ec713b4c0c"} Jul 07 18:49:56.114500 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:56.114481 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tmlg4" event={"ID":"d305832c-a3ec-453e-9135-697b0b80c619","Type":"ContainerStarted","Data":"1fdd65a7de18e72b571eda641f46e07cd057039bf6eecd3c98ed62d6430bfd67"} Jul 07 18:49:56.115727 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:56.115699 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" event={"ID":"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55","Type":"ContainerStarted","Data":"202aec592fd367410d188772f80a9fac9afe2d3e252529df724d2bdbd23a028e"} Jul 07 18:49:56.116702 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:56.116685 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v6tvj" event={"ID":"ac705cb6-fdca-4ffe-b46b-7e299b498b3e","Type":"ContainerStarted","Data":"9feead14f826857903cf5b153ffec86a9ff91c48f2819db69598398cb295c2a6"} Jul 07 18:49:56.126992 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:56.126949 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-66.ec2.internal" podStartSLOduration=2.126938684 podStartE2EDuration="2.126938684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:49:56.126936284 +0000 UTC m=+3.546296508" watchObservedRunningTime="2026-07-07 18:49:56.126938684 +0000 UTC m=+3.546298907" Jul 07 18:49:56.605277 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:56.605179 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs\") pod \"network-metrics-daemon-hjnqt\" (UID: \"bcee6de1-03c6-4175-9e3b-c2f88c977d88\") " pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:49:56.605277 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:56.605257 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret\") pod \"global-pull-secret-syncer-hslvd\" (UID: \"2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf\") " pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:49:56.605479 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:56.605387 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:49:56.605479 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:56.605445 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret podName:2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf nodeName:}" failed. No retries permitted until 2026-07-07 18:49:58.605427998 +0000 UTC m=+6.024788201 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret") pod "global-pull-secret-syncer-hslvd" (UID: "2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:49:56.605939 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:56.605920 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:49:56.606033 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:56.605975 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs podName:bcee6de1-03c6-4175-9e3b-c2f88c977d88 nodeName:}" failed. No retries permitted until 2026-07-07 18:49:58.605959125 +0000 UTC m=+6.025319335 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs") pod "network-metrics-daemon-hjnqt" (UID: "bcee6de1-03c6-4175-9e3b-c2f88c977d88") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:49:56.674488 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:56.674210 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 18:49:56.705766 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:56.705663 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdhvc\" (UniqueName: \"kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc\") pod \"network-check-target-4hhc9\" (UID: \"6f83f082-dd3d-4223-b29b-f3c00bd060cd\") " pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:49:56.705890 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:56.705856 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:49:56.705890 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:56.705879 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:49:56.705890 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:56.705891 2564 projected.go:196] Error preparing data for projected volume kube-api-access-sdhvc for pod openshift-network-diagnostics/network-check-target-4hhc9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:49:56.706064 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:56.705944 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc podName:6f83f082-dd3d-4223-b29b-f3c00bd060cd nodeName:}" failed. No retries permitted until 2026-07-07 18:49:58.705927537 +0000 UTC m=+6.125287743 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-sdhvc" (UniqueName: "kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc") pod "network-check-target-4hhc9" (UID: "6f83f082-dd3d-4223-b29b-f3c00bd060cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:49:57.024684 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:57.024540 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 18:44:55 +0000 UTC" deadline="2028-02-07 19:49:58.917325048 +0000 UTC" Jul 07 18:49:57.024684 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:57.024611 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13921h0m1.892719066s" Jul 07 18:49:57.100594 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:57.100106 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:49:57.100594 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:57.100208 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjnqt" podUID="bcee6de1-03c6-4175-9e3b-c2f88c977d88" Jul 07 18:49:57.129675 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:57.129644 2564 generic.go:356] "Generic (PLEG): container finished" podID="b4755395f1b291c9d732ae38993723a3" containerID="30ca531faf0f62f2a1c6522ca87b3cee55350535b830084994c5027f7a69a3f2" exitCode=0 Jul 07 18:49:57.130502 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:57.130475 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-66.ec2.internal" event={"ID":"b4755395f1b291c9d732ae38993723a3","Type":"ContainerDied","Data":"30ca531faf0f62f2a1c6522ca87b3cee55350535b830084994c5027f7a69a3f2"} Jul 07 18:49:58.097670 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:58.097633 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:49:58.098086 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:58.097672 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:49:58.098086 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:58.097769 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hhc9" podUID="6f83f082-dd3d-4223-b29b-f3c00bd060cd" Jul 07 18:49:58.098086 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:58.097883 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hslvd" podUID="2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf" Jul 07 18:49:58.145100 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:58.145065 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-66.ec2.internal" event={"ID":"b4755395f1b291c9d732ae38993723a3","Type":"ContainerStarted","Data":"27b2aa0a591ce70c3c086fff5c5391c8a162d426e550a10b28fc692df931bfdc"} Jul 07 18:49:58.164926 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:58.164876 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-66.ec2.internal" podStartSLOduration=4.16485729 podStartE2EDuration="4.16485729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:49:58.163959956 +0000 UTC m=+5.583320177" watchObservedRunningTime="2026-07-07 18:49:58.16485729 +0000 UTC m=+5.584217516" Jul 07 18:49:58.623471 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:58.623439 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs\") pod \"network-metrics-daemon-hjnqt\" (UID: \"bcee6de1-03c6-4175-9e3b-c2f88c977d88\") " pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:49:58.623650 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:58.623518 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret\") pod \"global-pull-secret-syncer-hslvd\" (UID: \"2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf\") " pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:49:58.623650 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:58.623644 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:49:58.623771 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:58.623688 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret podName:2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf nodeName:}" failed. No retries permitted until 2026-07-07 18:50:02.623675621 +0000 UTC m=+10.043035829 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret") pod "global-pull-secret-syncer-hslvd" (UID: "2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:49:58.623964 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:58.623949 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:49:58.624022 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:58.623987 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs podName:bcee6de1-03c6-4175-9e3b-c2f88c977d88 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:02.62397673 +0000 UTC m=+10.043336932 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs") pod "network-metrics-daemon-hjnqt" (UID: "bcee6de1-03c6-4175-9e3b-c2f88c977d88") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:49:58.724371 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:58.724314 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdhvc\" (UniqueName: \"kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc\") pod \"network-check-target-4hhc9\" (UID: \"6f83f082-dd3d-4223-b29b-f3c00bd060cd\") " pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:49:58.724514 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:58.724495 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:49:58.724620 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:58.724518 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:49:58.724620 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:58.724531 2564 projected.go:196] Error preparing data for projected volume kube-api-access-sdhvc for pod openshift-network-diagnostics/network-check-target-4hhc9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:49:58.724620 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:58.724596 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc podName:6f83f082-dd3d-4223-b29b-f3c00bd060cd nodeName:}" failed. No retries permitted until 2026-07-07 18:50:02.724565447 +0000 UTC m=+10.143925651 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-sdhvc" (UniqueName: "kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc") pod "network-check-target-4hhc9" (UID: "6f83f082-dd3d-4223-b29b-f3c00bd060cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:49:59.100488 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:49:59.099991 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:49:59.100488 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:49:59.100124 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjnqt" podUID="bcee6de1-03c6-4175-9e3b-c2f88c977d88" Jul 07 18:50:00.097831 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:00.097797 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:00.098003 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:00.097927 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hslvd" podUID="2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf" Jul 07 18:50:00.098003 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:00.097984 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:50:00.098147 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:00.098103 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hhc9" podUID="6f83f082-dd3d-4223-b29b-f3c00bd060cd" Jul 07 18:50:01.098796 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:01.098763 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:50:01.099155 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:01.098906 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjnqt" podUID="bcee6de1-03c6-4175-9e3b-c2f88c977d88" Jul 07 18:50:02.097553 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:02.097519 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:50:02.097754 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:02.097667 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hhc9" podUID="6f83f082-dd3d-4223-b29b-f3c00bd060cd" Jul 07 18:50:02.098068 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:02.098044 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:02.098186 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:02.098137 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hslvd" podUID="2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf" Jul 07 18:50:02.659974 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:02.659939 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret\") pod \"global-pull-secret-syncer-hslvd\" (UID: \"2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf\") " pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:02.660309 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:02.660011 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs\") pod \"network-metrics-daemon-hjnqt\" (UID: \"bcee6de1-03c6-4175-9e3b-c2f88c977d88\") " pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:50:02.660309 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:02.660150 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:50:02.660309 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:02.660204 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs podName:bcee6de1-03c6-4175-9e3b-c2f88c977d88 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:10.660187231 +0000 UTC m=+18.079547438 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs") pod "network-metrics-daemon-hjnqt" (UID: "bcee6de1-03c6-4175-9e3b-c2f88c977d88") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:50:02.660526 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:02.660512 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:50:02.660569 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:02.660551 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret podName:2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf nodeName:}" failed. No retries permitted until 2026-07-07 18:50:10.660540887 +0000 UTC m=+18.079901090 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret") pod "global-pull-secret-syncer-hslvd" (UID: "2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:50:02.761172 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:02.761138 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdhvc\" (UniqueName: \"kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc\") pod \"network-check-target-4hhc9\" (UID: \"6f83f082-dd3d-4223-b29b-f3c00bd060cd\") " pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:50:02.761336 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:02.761322 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:50:02.761383 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:02.761345 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:50:02.761383 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:02.761357 2564 projected.go:196] Error preparing data for projected volume kube-api-access-sdhvc for pod openshift-network-diagnostics/network-check-target-4hhc9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:50:02.761476 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:02.761409 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc podName:6f83f082-dd3d-4223-b29b-f3c00bd060cd nodeName:}" failed. No retries permitted until 2026-07-07 18:50:10.761391001 +0000 UTC m=+18.180751207 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-sdhvc" (UniqueName: "kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc") pod "network-check-target-4hhc9" (UID: "6f83f082-dd3d-4223-b29b-f3c00bd060cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:50:03.098759 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:03.098649 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:50:03.098898 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:03.098765 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjnqt" podUID="bcee6de1-03c6-4175-9e3b-c2f88c977d88" Jul 07 18:50:04.097408 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:04.096969 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:50:04.097408 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:04.097079 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hhc9" podUID="6f83f082-dd3d-4223-b29b-f3c00bd060cd" Jul 07 18:50:04.097408 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:04.097173 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:04.097408 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:04.097269 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hslvd" podUID="2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf" Jul 07 18:50:05.097245 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:05.097220 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:50:05.097379 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:05.097319 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjnqt" podUID="bcee6de1-03c6-4175-9e3b-c2f88c977d88" Jul 07 18:50:06.097513 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:06.097478 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:50:06.097960 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:06.097478 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:06.097960 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:06.097609 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hhc9" podUID="6f83f082-dd3d-4223-b29b-f3c00bd060cd" Jul 07 18:50:06.097960 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:06.097706 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hslvd" podUID="2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf" Jul 07 18:50:07.097151 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:07.097114 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:50:07.097312 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:07.097266 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjnqt" podUID="bcee6de1-03c6-4175-9e3b-c2f88c977d88" Jul 07 18:50:08.097127 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:08.097102 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:08.097127 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:08.097137 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:50:08.097643 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:08.097242 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hslvd" podUID="2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf" Jul 07 18:50:08.097643 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:08.097308 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hhc9" podUID="6f83f082-dd3d-4223-b29b-f3c00bd060cd" Jul 07 18:50:09.097223 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:09.097195 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:50:09.097704 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:09.097306 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjnqt" podUID="bcee6de1-03c6-4175-9e3b-c2f88c977d88" Jul 07 18:50:10.097988 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:10.097952 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:10.097988 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:10.097985 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:50:10.098520 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:10.098089 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hslvd" podUID="2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf" Jul 07 18:50:10.098520 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:10.098216 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hhc9" podUID="6f83f082-dd3d-4223-b29b-f3c00bd060cd" Jul 07 18:50:10.722272 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:10.722198 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs\") pod \"network-metrics-daemon-hjnqt\" (UID: \"bcee6de1-03c6-4175-9e3b-c2f88c977d88\") " pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:50:10.722272 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:10.722270 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret\") pod \"global-pull-secret-syncer-hslvd\" (UID: \"2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf\") " pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:10.722447 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:10.722346 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:50:10.722447 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:10.722373 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:50:10.722447 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:10.722414 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs podName:bcee6de1-03c6-4175-9e3b-c2f88c977d88 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:26.722395637 +0000 UTC m=+34.141755854 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs") pod "network-metrics-daemon-hjnqt" (UID: "bcee6de1-03c6-4175-9e3b-c2f88c977d88") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:50:10.722447 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:10.722434 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret podName:2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf nodeName:}" failed. No retries permitted until 2026-07-07 18:50:26.722426245 +0000 UTC m=+34.141786452 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret") pod "global-pull-secret-syncer-hslvd" (UID: "2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:50:10.823517 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:10.823481 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdhvc\" (UniqueName: \"kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc\") pod \"network-check-target-4hhc9\" (UID: \"6f83f082-dd3d-4223-b29b-f3c00bd060cd\") " pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:50:10.823685 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:10.823640 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:50:10.823685 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:10.823662 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:50:10.823685 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:10.823670 2564 projected.go:196] Error preparing data for projected volume kube-api-access-sdhvc for pod openshift-network-diagnostics/network-check-target-4hhc9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:50:10.823819 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:10.823731 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc podName:6f83f082-dd3d-4223-b29b-f3c00bd060cd nodeName:}" failed. No retries permitted until 2026-07-07 18:50:26.82371137 +0000 UTC m=+34.243071589 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-sdhvc" (UniqueName: "kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc") pod "network-check-target-4hhc9" (UID: "6f83f082-dd3d-4223-b29b-f3c00bd060cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:50:11.097832 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:11.097744 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:50:11.097987 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:11.097883 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjnqt" podUID="bcee6de1-03c6-4175-9e3b-c2f88c977d88" Jul 07 18:50:12.096972 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:12.096940 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:50:12.097380 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:12.097065 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hhc9" podUID="6f83f082-dd3d-4223-b29b-f3c00bd060cd" Jul 07 18:50:12.097380 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:12.097121 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:12.097380 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:12.097210 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hslvd" podUID="2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf" Jul 07 18:50:13.098441 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:13.098409 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:50:13.098941 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:13.098529 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjnqt" podUID="bcee6de1-03c6-4175-9e3b-c2f88c977d88" Jul 07 18:50:14.097086 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:14.097051 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:14.097276 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:14.097051 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:50:14.097276 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:14.097209 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hslvd" podUID="2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf" Jul 07 18:50:14.097276 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:14.097263 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hhc9" podUID="6f83f082-dd3d-4223-b29b-f3c00bd060cd" Jul 07 18:50:15.097812 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:15.097550 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:50:15.098275 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:15.097945 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjnqt" podUID="bcee6de1-03c6-4175-9e3b-c2f88c977d88" Jul 07 18:50:15.178272 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:15.178116 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" event={"ID":"d21cfcc6-7fde-4272-86ff-26031eeacbda","Type":"ContainerStarted","Data":"c277f80508be558baf165f13324829f97d6b38754b67125fb1dc5bb115755b16"} Jul 07 18:50:15.179976 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:15.179954 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" event={"ID":"52108ccf-1379-48c0-8477-5ab8914619dc","Type":"ContainerStarted","Data":"4b3d2b1d050afc9bca83f8925a2d02c91d8f625b43d981320396927acc01c15b"} Jul 07 18:50:15.180093 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:15.180023 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" event={"ID":"52108ccf-1379-48c0-8477-5ab8914619dc","Type":"ContainerStarted","Data":"7d476857c77a23cbcd9ff4ca61bda26c3ad52c05a9936febf3f4ed42ed00a3fc"} Jul 07 18:50:15.181249 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:15.181226 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tmlg4" event={"ID":"d305832c-a3ec-453e-9135-697b0b80c619","Type":"ContainerStarted","Data":"47175c3810a2d4d6fa194a9be51e4dcd8f0690455788021d1d696f5407cce029"} Jul 07 18:50:15.182839 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:15.182800 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" event={"ID":"28c0d463-ad7d-4ae5-b801-1dbe90cb7c55","Type":"ContainerStarted","Data":"2250f2181b71b5c7a3665c25f725d079424996ff5ecd517d6aeb43c98d63ae77"} Jul 07 18:50:15.184901 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:15.184862 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v6tvj" event={"ID":"ac705cb6-fdca-4ffe-b46b-7e299b498b3e","Type":"ContainerStarted","Data":"3f738d8f3dc6bb1dd24f83d193927ec1880502c6750345250dbc8a9f8769da4e"} Jul 07 18:50:15.186315 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:15.186293 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dg4c4" event={"ID":"217e2ad4-6057-41e0-a4e8-3d2bd461fd27","Type":"ContainerStarted","Data":"b99d2cb662ea6191b6c9b80b1f70c7ce4db90b810638a2cb1fd3fa1adb8bb7df"} Jul 07 18:50:15.187761 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:15.187739 2564 generic.go:356] "Generic (PLEG): container finished" podID="09b26336-e7e1-40b3-b55c-f2640139c685" containerID="3b1c0c3bb109ebae09267a3ee65c28d1b7bee434140c84fd6da35e060061184f" exitCode=0 Jul 07 18:50:15.187837 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:15.187820 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8pkcb" event={"ID":"09b26336-e7e1-40b3-b55c-f2640139c685","Type":"ContainerDied","Data":"3b1c0c3bb109ebae09267a3ee65c28d1b7bee434140c84fd6da35e060061184f"} Jul 07 18:50:15.189807 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:15.189784 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vw7p6" event={"ID":"4b1d9d37-d1f8-4ebd-b96d-0a11e50a90e4","Type":"ContainerStarted","Data":"21dc12f5f5cfe3ccf318b515a898f455361ee655b18089a7b2f7ea870b5de144"} Jul 07 18:50:15.199426 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:15.199391 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-tmlg4" podStartSLOduration=3.332130113 podStartE2EDuration="22.199379543s" totalImagesPullingTime="18.86724943s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:15.198957641 +0000 UTC m=+22.618317865" watchObservedRunningTime="2026-07-07 18:50:15.199379543 +0000 UTC m=+22.618739765" Jul 07 18:50:15.250550 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:15.250512 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dg4c4" podStartSLOduration=3.339580248 podStartE2EDuration="22.2504889s" totalImagesPullingTime="18.910908652s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:15.249957426 +0000 UTC m=+22.669317649" watchObservedRunningTime="2026-07-07 18:50:15.2504889 +0000 UTC m=+22.669849121" Jul 07 18:50:15.250959 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:15.250936 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cnvgc" podStartSLOduration=3.38106653 podStartE2EDuration="22.250930318s" totalImagesPullingTime="18.869863788s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:15.222100514 +0000 UTC m=+22.641460740" watchObservedRunningTime="2026-07-07 18:50:15.250930318 +0000 UTC m=+22.670290542" Jul 07 18:50:15.270612 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:15.270553 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vw7p6" podStartSLOduration=7.740325262 podStartE2EDuration="22.270540922s" totalImagesPullingTime="14.53021566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:49:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:15.270342702 +0000 UTC m=+22.689702926" watchObservedRunningTime="2026-07-07 18:50:15.270540922 +0000 UTC m=+22.689901145" Jul 07 18:50:15.320662 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:15.320625 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-v6tvj" podStartSLOduration=3.432261106 podStartE2EDuration="22.32061265s" totalImagesPullingTime="18.888351544s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:15.296632152 +0000 UTC m=+22.715992376" watchObservedRunningTime="2026-07-07 18:50:15.32061265 +0000 UTC m=+22.739972873" Jul 07 18:50:15.684343 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:15.684295 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vw7p6" Jul 07 18:50:15.684777 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:15.684764 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vw7p6" Jul 07 18:50:16.097653 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:16.097624 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:50:16.097781 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:16.097661 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:16.097781 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:16.097733 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hhc9" podUID="6f83f082-dd3d-4223-b29b-f3c00bd060cd" Jul 07 18:50:16.098339 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:16.097875 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hslvd" podUID="2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf" Jul 07 18:50:16.195942 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:16.195908 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" event={"ID":"52108ccf-1379-48c0-8477-5ab8914619dc","Type":"ContainerStarted","Data":"7aeb484208068ef028ffec2c5af4bff8f07386db9d25b2bdce62a1aa63696357"} Jul 07 18:50:16.196134 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:16.195995 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" event={"ID":"52108ccf-1379-48c0-8477-5ab8914619dc","Type":"ContainerStarted","Data":"0e25754dee8f40449b1bbdca5c8a9c1bc36b6caa30e7b65cac9daed717be4700"} Jul 07 18:50:16.196134 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:16.196042 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" event={"ID":"52108ccf-1379-48c0-8477-5ab8914619dc","Type":"ContainerStarted","Data":"f9933d2deee24716880cc82e6c5d6ba9a0676061ce9e6498433a63ca74fca1b0"} Jul 07 18:50:16.196134 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:16.196079 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" event={"ID":"52108ccf-1379-48c0-8477-5ab8914619dc","Type":"ContainerStarted","Data":"6689db6b7fa0beb945e63cdaf11e0cc462db45c016556308c052bc7cc99b3516"} Jul 07 18:50:16.198688 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:16.198654 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rq4nf" event={"ID":"50f2f138-aebd-48b3-92eb-87c35347e4c6","Type":"ContainerStarted","Data":"689f6621cb02cfadd868c0fb01f4b580edaef90cf754cea4200a88519f0cdae6"} Jul 07 18:50:16.198808 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:16.198749 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vw7p6" Jul 07 18:50:16.199172 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:16.199128 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vw7p6" Jul 07 18:50:16.216906 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:16.216861 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rq4nf" podStartSLOduration=4.360631988 podStartE2EDuration="23.216849395s" totalImagesPullingTime="18.856217407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:16.216459073 +0000 UTC m=+23.635819298" watchObservedRunningTime="2026-07-07 18:50:16.216849395 +0000 UTC m=+23.636209653" Jul 07 18:50:16.672206 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:16.672188 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 18:50:17.058394 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:17.058296 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T18:50:16.672202471Z","UUID":"4741d8ab-f305-42a2-b72a-a8fea3505c3f","Handler":null,"Name":"","Endpoint":""} Jul 07 18:50:17.059867 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:17.059846 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 18:50:17.060011 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:17.059888 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 18:50:17.097174 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:17.097152 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:50:17.097320 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:17.097259 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjnqt" podUID="bcee6de1-03c6-4175-9e3b-c2f88c977d88" Jul 07 18:50:17.202330 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:17.202280 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" event={"ID":"d21cfcc6-7fde-4272-86ff-26031eeacbda","Type":"ContainerStarted","Data":"f7d761e3572e8e52462678788bffd93369e143959406bcfb623b2e63f2f7b896"} Jul 07 18:50:18.096928 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:18.096905 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:50:18.096928 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:18.096914 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:18.097060 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:18.096991 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hhc9" podUID="6f83f082-dd3d-4223-b29b-f3c00bd060cd" Jul 07 18:50:18.097118 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:18.097102 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hslvd" podUID="2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf" Jul 07 18:50:18.205747 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:18.205687 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" event={"ID":"d21cfcc6-7fde-4272-86ff-26031eeacbda","Type":"ContainerStarted","Data":"592694798e3e2d903cbebcce81509eca1a6cfdb1e426a0165d98652bc6b790f7"} Jul 07 18:50:18.208943 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:18.208922 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" event={"ID":"52108ccf-1379-48c0-8477-5ab8914619dc","Type":"ContainerStarted","Data":"043ff6e1e8631f7d89fc5f40b73e62e05707a3f974a093959612a8cf47ca77d6"} Jul 07 18:50:18.231374 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:18.231331 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mh5jg" podStartSLOduration=3.868185324 podStartE2EDuration="25.231320081s" totalImagesPullingTime="21.363134757s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:18.230838658 +0000 UTC m=+25.650198884" watchObservedRunningTime="2026-07-07 18:50:18.231320081 +0000 UTC m=+25.650680304" Jul 07 18:50:19.097854 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:19.097651 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:50:19.098031 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:19.097943 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjnqt" podUID="bcee6de1-03c6-4175-9e3b-c2f88c977d88" Jul 07 18:50:20.097034 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:20.096961 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:50:20.097034 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:20.096999 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:20.097466 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:20.097093 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hhc9" podUID="6f83f082-dd3d-4223-b29b-f3c00bd060cd" Jul 07 18:50:20.097466 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:20.097240 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hslvd" podUID="2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf" Jul 07 18:50:21.098340 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:21.097341 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:50:21.098340 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:21.097471 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjnqt" podUID="bcee6de1-03c6-4175-9e3b-c2f88c977d88" Jul 07 18:50:21.220306 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:21.220030 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" event={"ID":"52108ccf-1379-48c0-8477-5ab8914619dc","Type":"ContainerStarted","Data":"4ff8a3afd51cf1680a689cadd8352d50e80ab2173420f54dc4eb322128272346"} Jul 07 18:50:21.220473 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:21.220367 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:50:21.222224 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:21.222197 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8pkcb" event={"ID":"09b26336-e7e1-40b3-b55c-f2640139c685","Type":"ContainerStarted","Data":"08379fef972b7730018bba8f2e8ed9b9538f05becfd523319a1f5a9b9a03c30d"} Jul 07 18:50:21.236094 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:21.236075 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:50:21.256211 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:21.256163 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" podStartSLOduration=9.248664904 podStartE2EDuration="28.256149788s" totalImagesPullingTime="19.007484884s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:21.25575635 +0000 UTC m=+28.675116595" watchObservedRunningTime="2026-07-07 18:50:21.256149788 +0000 UTC m=+28.675510011" Jul 07 18:50:22.097503 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:22.097474 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:50:22.097782 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:22.097587 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hhc9" podUID="6f83f082-dd3d-4223-b29b-f3c00bd060cd" Jul 07 18:50:22.097782 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:22.097603 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:22.097782 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:22.097690 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hslvd" podUID="2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf" Jul 07 18:50:22.226190 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:22.226160 2564 generic.go:356] "Generic (PLEG): container finished" podID="09b26336-e7e1-40b3-b55c-f2640139c685" containerID="08379fef972b7730018bba8f2e8ed9b9538f05becfd523319a1f5a9b9a03c30d" exitCode=0 Jul 07 18:50:22.226597 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:22.226236 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8pkcb" event={"ID":"09b26336-e7e1-40b3-b55c-f2640139c685","Type":"ContainerDied","Data":"08379fef972b7730018bba8f2e8ed9b9538f05becfd523319a1f5a9b9a03c30d"} Jul 07 18:50:22.226867 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:22.226849 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:50:22.226963 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:22.226936 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:50:22.242003 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:22.241984 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:50:23.062092 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:23.062064 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hjnqt"] Jul 07 18:50:23.062264 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:23.062184 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:50:23.062405 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:23.062382 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjnqt" podUID="bcee6de1-03c6-4175-9e3b-c2f88c977d88" Jul 07 18:50:23.065047 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:23.065019 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hslvd"] Jul 07 18:50:23.065161 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:23.065104 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:23.065217 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:23.065198 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hslvd" podUID="2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf" Jul 07 18:50:23.065702 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:23.065684 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4hhc9"] Jul 07 18:50:23.065771 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:23.065747 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:50:23.065846 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:23.065828 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hhc9" podUID="6f83f082-dd3d-4223-b29b-f3c00bd060cd" Jul 07 18:50:24.233671 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:24.233586 2564 generic.go:356] "Generic (PLEG): container finished" podID="09b26336-e7e1-40b3-b55c-f2640139c685" containerID="be897eb9c403b73539dac419798fee3d8004b29a3b0b7eaee976773a938db51b" exitCode=0 Jul 07 18:50:24.233671 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:24.233643 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8pkcb" event={"ID":"09b26336-e7e1-40b3-b55c-f2640139c685","Type":"ContainerDied","Data":"be897eb9c403b73539dac419798fee3d8004b29a3b0b7eaee976773a938db51b"} Jul 07 18:50:25.097994 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:25.097842 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:50:25.098094 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:25.097897 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:25.098158 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:25.098085 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hhc9" podUID="6f83f082-dd3d-4223-b29b-f3c00bd060cd" Jul 07 18:50:25.098158 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:25.098131 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hslvd" podUID="2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf" Jul 07 18:50:25.098158 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:25.097933 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:50:25.098311 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:25.098223 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjnqt" podUID="bcee6de1-03c6-4175-9e3b-c2f88c977d88" Jul 07 18:50:25.239907 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:25.239842 2564 generic.go:356] "Generic (PLEG): container finished" podID="09b26336-e7e1-40b3-b55c-f2640139c685" containerID="0651cc7a410f6bad82a2e56a4624481058f34210b31a81886fbc2df9b7b8c55b" exitCode=0 Jul 07 18:50:25.239907 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:25.239882 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8pkcb" event={"ID":"09b26336-e7e1-40b3-b55c-f2640139c685","Type":"ContainerDied","Data":"0651cc7a410f6bad82a2e56a4624481058f34210b31a81886fbc2df9b7b8c55b"} Jul 07 18:50:26.743569 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:26.743495 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs\") pod \"network-metrics-daemon-hjnqt\" (UID: \"bcee6de1-03c6-4175-9e3b-c2f88c977d88\") " pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:50:26.743569 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:26.743545 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret\") pod \"global-pull-secret-syncer-hslvd\" (UID: \"2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf\") " pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:26.744062 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:26.743669 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:50:26.744062 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:26.743744 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs podName:bcee6de1-03c6-4175-9e3b-c2f88c977d88 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:58.743724541 +0000 UTC m=+66.163084744 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs") pod "network-metrics-daemon-hjnqt" (UID: "bcee6de1-03c6-4175-9e3b-c2f88c977d88") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:50:26.744062 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:26.743767 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:50:26.744062 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:26.743847 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret podName:2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf nodeName:}" failed. No retries permitted until 2026-07-07 18:50:58.743830519 +0000 UTC m=+66.163190726 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret") pod "global-pull-secret-syncer-hslvd" (UID: "2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:50:26.844770 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:26.844737 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdhvc\" (UniqueName: \"kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc\") pod \"network-check-target-4hhc9\" (UID: \"6f83f082-dd3d-4223-b29b-f3c00bd060cd\") " pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:50:26.844911 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:26.844889 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:50:26.844911 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:26.844904 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:50:26.845009 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:26.844913 2564 projected.go:196] Error preparing data for projected volume kube-api-access-sdhvc for pod openshift-network-diagnostics/network-check-target-4hhc9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:50:26.845009 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:26.844959 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc podName:6f83f082-dd3d-4223-b29b-f3c00bd060cd nodeName:}" failed. No retries permitted until 2026-07-07 18:50:58.844946393 +0000 UTC m=+66.264306612 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-sdhvc" (UniqueName: "kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc") pod "network-check-target-4hhc9" (UID: "6f83f082-dd3d-4223-b29b-f3c00bd060cd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:50:27.098035 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:27.097961 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:27.098171 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:27.098076 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:50:27.098171 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:27.098076 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hslvd" podUID="2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf" Jul 07 18:50:27.098171 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:27.098143 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4hhc9" podUID="6f83f082-dd3d-4223-b29b-f3c00bd060cd" Jul 07 18:50:27.098171 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:27.097961 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:50:27.098329 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:27.098231 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hjnqt" podUID="bcee6de1-03c6-4175-9e3b-c2f88c977d88" Jul 07 18:50:28.931479 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.931449 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-66.ec2.internal" event="NodeReady" Jul 07 18:50:28.931921 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.931554 2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 18:50:28.977589 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.977541 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-qx6d9"] Jul 07 18:50:28.979316 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.979299 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" Jul 07 18:50:28.980833 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.980810 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-vcwql"] Jul 07 18:50:28.982589 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.982556 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6c64b8fbfb-nfsfs"] Jul 07 18:50:28.983437 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.982664 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-vcwql" Jul 07 18:50:28.983719 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.983684 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 07 18:50:28.983719 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.983697 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 07 18:50:28.983854 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.983741 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-bwcqg\"" Jul 07 18:50:28.983854 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.983753 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 18:50:28.984007 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.983994 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 18:50:28.984244 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.984224 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk9lb"] Jul 07 18:50:28.984357 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.984304 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:28.985901 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.985883 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-55mgr"] Jul 07 18:50:28.985998 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.985988 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk9lb" Jul 07 18:50:28.986454 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.986436 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:50:28.986454 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.986447 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 07 18:50:28.987687 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.987667 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl"] Jul 07 18:50:28.987770 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.987724 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-55mgr" Jul 07 18:50:28.988364 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.988347 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wb95j\"" Jul 07 18:50:28.988483 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.988466 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 07 18:50:28.989161 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.989144 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 07 18:50:28.989334 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.989309 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 18:50:28.989504 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.989487 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 18:50:28.989599 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.989513 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 07 18:50:28.989665 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.989634 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-x4rt9\"" Jul 07 18:50:28.989722 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.989566 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-whczq\"" Jul 07 18:50:28.989972 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.989954 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl" Jul 07 18:50:28.990051 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.989998 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 07 18:50:28.990051 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.990031 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 07 18:50:28.990827 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.990809 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:50:28.990912 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.990834 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:50:28.990969 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.990913 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 18:50:28.990969 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.990924 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-zs8z4\"" Jul 07 18:50:28.992351 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.992331 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 07 18:50:28.993464 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.993437 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 07 18:50:28.993823 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.993803 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 18:50:28.993930 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.993862 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 07 18:50:28.993930 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.993892 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 18:50:28.994484 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.994464 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cnmmb\"" Jul 07 18:50:28.994629 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.994608 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-blnhp"] Jul 07 18:50:28.994806 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.994792 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 07 18:50:28.996554 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.996537 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7dd847dbd5-mq8mz"] Jul 07 18:50:28.996689 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.996672 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-blnhp" Jul 07 18:50:28.999893 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:28.999867 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-nm9rv\"" Jul 07 18:50:29.002327 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.001681 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 07 18:50:29.002327 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.002306 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:29.005206 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.004542 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 07 18:50:29.005206 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.004780 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 07 18:50:29.005740 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.005623 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 07 18:50:29.005838 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.005632 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:50:29.006689 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.006652 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-c4ql5"] Jul 07 18:50:29.008862 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.008841 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 18:50:29.009106 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.009090 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 07 18:50:29.009270 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.009252 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 07 18:50:29.009371 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.009310 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 07 18:50:29.009486 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.009468 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 07 18:50:29.010737 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.010718 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 18:50:29.011146 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.011131 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-qx6d9"] Jul 07 18:50:29.011472 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.011447 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-c4ql5" Jul 07 18:50:29.011915 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.011897 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk9lb"] Jul 07 18:50:29.012484 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.012465 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-vcwql"] Jul 07 18:50:29.013231 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.013213 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-55mgr"] Jul 07 18:50:29.013775 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.013756 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-mq4f7"] Jul 07 18:50:29.015656 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.015639 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 18:50:29.015829 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.015747 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c64b8fbfb-nfsfs"] Jul 07 18:50:29.015895 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.015864 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-blnhp"] Jul 07 18:50:29.015950 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.015907 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mq4f7" Jul 07 18:50:29.015950 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.015913 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 07 18:50:29.015950 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.015941 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4qmc4\"" Jul 07 18:50:29.016204 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.016177 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:50:29.016329 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.016313 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-c4ql5"] Jul 07 18:50:29.016404 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.016295 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 07 18:50:29.016404 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.016396 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-z5dxq\"" Jul 07 18:50:29.017007 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.016989 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl"] Jul 07 18:50:29.020121 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.020104 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7dd847dbd5-mq8mz"] Jul 07 18:50:29.021320 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.021300 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-mq4f7"] Jul 07 18:50:29.022839 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.021648 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 18:50:29.022839 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.022097 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-dbg2z\"" Jul 07 18:50:29.022839 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.022550 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 18:50:29.079588 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.072862 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7a3286ca-3966-48e7-a4d9-9830a7fb1063-serving-cert\") pod \"console-operator-68b757fb67-vcwql\" (UID: \"7a3286ca-3966-48e7-a4d9-9830a7fb1063\") " pod="openshift-console-operator/console-operator-68b757fb67-vcwql" Jul 07 18:50:29.079588 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.072901 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c3b0a294-e7a3-49fa-a089-92d4bc757e8e-serving-cert\") pod \"insights-operator-5c847dcfc4-qx6d9\" (UID: \"c3b0a294-e7a3-49fa-a089-92d4bc757e8e\") " pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" Jul 07 18:50:29.079588 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.072927 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.079588 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.072973 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7a3286ca-3966-48e7-a4d9-9830a7fb1063-config\") pod \"console-operator-68b757fb67-vcwql\" (UID: \"7a3286ca-3966-48e7-a4d9-9830a7fb1063\") " pod="openshift-console-operator/console-operator-68b757fb67-vcwql" Jul 07 18:50:29.079588 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.072995 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfkrx\" (UniqueName: \"kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-kube-api-access-gfkrx\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.079588 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.073016 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7a3286ca-3966-48e7-a4d9-9830a7fb1063-trusted-ca\") pod \"console-operator-68b757fb67-vcwql\" (UID: \"7a3286ca-3966-48e7-a4d9-9830a7fb1063\") " pod="openshift-console-operator/console-operator-68b757fb67-vcwql" Jul 07 18:50:29.079588 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.073059 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/793c0cea-e51b-4fe8-89ba-425492267b6a-trusted-ca\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.079588 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.073082 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3b0a294-e7a3-49fa-a089-92d4bc757e8e-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-qx6d9\" (UID: \"c3b0a294-e7a3-49fa-a089-92d4bc757e8e\") " pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" Jul 07 18:50:29.079588 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.073115 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/793c0cea-e51b-4fe8-89ba-425492267b6a-image-registry-private-configuration\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.079588 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.073140 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/793c0cea-e51b-4fe8-89ba-425492267b6a-ca-trust-extracted\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.079588 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.073164 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-bound-sa-token\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.079588 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.073207 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-certificates\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.079588 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.073233 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrjm7\" (UniqueName: \"kubernetes.io/projected/7a3286ca-3966-48e7-a4d9-9830a7fb1063-kube-api-access-jrjm7\") pod \"console-operator-68b757fb67-vcwql\" (UID: \"7a3286ca-3966-48e7-a4d9-9830a7fb1063\") " pod="openshift-console-operator/console-operator-68b757fb67-vcwql" Jul 07 18:50:29.079588 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.073255 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3b0a294-e7a3-49fa-a089-92d4bc757e8e-tmp\") pod \"insights-operator-5c847dcfc4-qx6d9\" (UID: \"c3b0a294-e7a3-49fa-a089-92d4bc757e8e\") " pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" Jul 07 18:50:29.079588 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.073277 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c3b0a294-e7a3-49fa-a089-92d4bc757e8e-snapshots\") pod \"insights-operator-5c847dcfc4-qx6d9\" (UID: \"c3b0a294-e7a3-49fa-a089-92d4bc757e8e\") " pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" Jul 07 18:50:29.080230 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.073300 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2282n\" (UniqueName: \"kubernetes.io/projected/c3b0a294-e7a3-49fa-a089-92d4bc757e8e-kube-api-access-2282n\") pod \"insights-operator-5c847dcfc4-qx6d9\" (UID: \"c3b0a294-e7a3-49fa-a089-92d4bc757e8e\") " pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" Jul 07 18:50:29.080230 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.073344 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/793c0cea-e51b-4fe8-89ba-425492267b6a-installation-pull-secrets\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.080230 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.073367 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3b0a294-e7a3-49fa-a089-92d4bc757e8e-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-qx6d9\" (UID: \"c3b0a294-e7a3-49fa-a089-92d4bc757e8e\") " pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" Jul 07 18:50:29.080230 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.077477 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wvsd8"] Jul 07 18:50:29.080917 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.080522 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wvsd8" Jul 07 18:50:29.083188 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.083010 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 18:50:29.083188 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.083026 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 18:50:29.083327 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.083271 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b4nbt\"" Jul 07 18:50:29.090784 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.090726 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wvsd8"] Jul 07 18:50:29.097026 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.096961 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:50:29.097115 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.097075 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:50:29.097190 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.096965 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:29.099388 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.099345 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l98hl\"" Jul 07 18:50:29.099480 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.099391 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cxlw6\"" Jul 07 18:50:29.099704 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.099690 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 18:50:29.100597 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.100277 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 18:50:29.141114 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.141090 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9s59k"] Jul 07 18:50:29.143263 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.143245 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9s59k" Jul 07 18:50:29.145884 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.145735 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 18:50:29.145884 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.145797 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 18:50:29.145884 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.145827 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-45jgr\"" Jul 07 18:50:29.145884 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.145821 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 18:50:29.152198 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.152179 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9s59k"] Jul 07 18:50:29.173930 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.173907 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/793c0cea-e51b-4fe8-89ba-425492267b6a-installation-pull-secrets\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.174042 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.173943 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3b0a294-e7a3-49fa-a089-92d4bc757e8e-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-qx6d9\" (UID: \"c3b0a294-e7a3-49fa-a089-92d4bc757e8e\") " pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" Jul 07 18:50:29.174042 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.173970 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/10350357-c651-4b89-8497-7e16a1b6c701-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-b5nnl\" (UID: \"10350357-c651-4b89-8497-7e16a1b6c701\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl" Jul 07 18:50:29.174042 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.173996 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf822\" (UniqueName: \"kubernetes.io/projected/1ce43d53-3718-49e6-a60e-ce5ca36877ca-kube-api-access-wf822\") pod \"kube-storage-version-migrator-operator-846bf668f7-blnhp\" (UID: \"1ce43d53-3718-49e6-a60e-ce5ca36877ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-blnhp" Jul 07 18:50:29.174042 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174032 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.174232 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174055 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qd4j\" (UniqueName: \"kubernetes.io/projected/23e1f849-e113-4a42-bc58-448388e582ce-kube-api-access-5qd4j\") pod \"dns-default-wvsd8\" (UID: \"23e1f849-e113-4a42-bc58-448388e582ce\") " pod="openshift-dns/dns-default-wvsd8" Jul 07 18:50:29.174232 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174102 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10350357-c651-4b89-8497-7e16a1b6c701-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-b5nnl\" (UID: \"10350357-c651-4b89-8497-7e16a1b6c701\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl" Jul 07 18:50:29.174232 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174129 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfkrx\" (UniqueName: \"kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-kube-api-access-gfkrx\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.174232 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.174143 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 18:50:29.174232 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174151 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ce43d53-3718-49e6-a60e-ce5ca36877ca-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-blnhp\" (UID: \"1ce43d53-3718-49e6-a60e-ce5ca36877ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-blnhp" Jul 07 18:50:29.174232 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.174157 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c64b8fbfb-nfsfs: secret "image-registry-tls" not found Jul 07 18:50:29.174232 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174184 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-default-certificate\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:29.174232 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174213 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23e1f849-e113-4a42-bc58-448388e582ce-metrics-tls\") pod \"dns-default-wvsd8\" (UID: \"23e1f849-e113-4a42-bc58-448388e582ce\") " pod="openshift-dns/dns-default-wvsd8" Jul 07 18:50:29.174624 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174249 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/793c0cea-e51b-4fe8-89ba-425492267b6a-trusted-ca\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.174624 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174271 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3b0a294-e7a3-49fa-a089-92d4bc757e8e-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-qx6d9\" (UID: \"c3b0a294-e7a3-49fa-a089-92d4bc757e8e\") " pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" Jul 07 18:50:29.174624 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174297 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-stats-auth\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:29.174624 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174323 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/793c0cea-e51b-4fe8-89ba-425492267b6a-image-registry-private-configuration\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.174624 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174349 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvzsd\" (UniqueName: \"kubernetes.io/projected/d6822119-b2f7-4a01-9738-66ee52dac067-kube-api-access-kvzsd\") pod \"cluster-samples-operator-56fcf86bf-c4ql5\" (UID: \"d6822119-b2f7-4a01-9738-66ee52dac067\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-c4ql5" Jul 07 18:50:29.174624 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174371 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23e1f849-e113-4a42-bc58-448388e582ce-tmp-dir\") pod \"dns-default-wvsd8\" (UID: \"23e1f849-e113-4a42-bc58-448388e582ce\") " pod="openshift-dns/dns-default-wvsd8" Jul 07 18:50:29.174624 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174397 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-certificates\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.174624 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.174504 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls podName:793c0cea-e51b-4fe8-89ba-425492267b6a nodeName:}" failed. No retries permitted until 2026-07-07 18:50:29.674482808 +0000 UTC m=+37.093843024 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls") pod "image-registry-6c64b8fbfb-nfsfs" (UID: "793c0cea-e51b-4fe8-89ba-425492267b6a") : secret "image-registry-tls" not found Jul 07 18:50:29.174624 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174564 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2k6l\" (UniqueName: \"kubernetes.io/projected/10350357-c651-4b89-8497-7e16a1b6c701-kube-api-access-r2k6l\") pod \"cluster-monitoring-operator-64786d9cbb-b5nnl\" (UID: \"10350357-c651-4b89-8497-7e16a1b6c701\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl" Jul 07 18:50:29.174624 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174622 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3b0a294-e7a3-49fa-a089-92d4bc757e8e-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-qx6d9\" (UID: \"c3b0a294-e7a3-49fa-a089-92d4bc757e8e\") " pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" Jul 07 18:50:29.175078 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174622 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmb2k\" (UniqueName: \"kubernetes.io/projected/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-kube-api-access-xmb2k\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:29.175078 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174680 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrjm7\" (UniqueName: \"kubernetes.io/projected/7a3286ca-3966-48e7-a4d9-9830a7fb1063-kube-api-access-jrjm7\") pod \"console-operator-68b757fb67-vcwql\" (UID: \"7a3286ca-3966-48e7-a4d9-9830a7fb1063\") " pod="openshift-console-operator/console-operator-68b757fb67-vcwql" Jul 07 18:50:29.175078 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174707 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3b0a294-e7a3-49fa-a089-92d4bc757e8e-tmp\") pod \"insights-operator-5c847dcfc4-qx6d9\" (UID: \"c3b0a294-e7a3-49fa-a089-92d4bc757e8e\") " pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" Jul 07 18:50:29.175078 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174738 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz6bw\" (UniqueName: \"kubernetes.io/projected/f868dc34-c7da-4e41-abd3-8ad56bae2bc6-kube-api-access-jz6bw\") pod \"network-check-source-989f8b8dc-mq4f7\" (UID: \"f868dc34-c7da-4e41-abd3-8ad56bae2bc6\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mq4f7" Jul 07 18:50:29.175078 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174782 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9c701b97-bb01-4f1b-af53-8f310ec406ca-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-wk9lb\" (UID: \"9c701b97-bb01-4f1b-af53-8f310ec406ca\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk9lb" Jul 07 18:50:29.175078 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.174817 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7a3286ca-3966-48e7-a4d9-9830a7fb1063-serving-cert\") pod \"console-operator-68b757fb67-vcwql\" (UID: \"7a3286ca-3966-48e7-a4d9-9830a7fb1063\") " pod="openshift-console-operator/console-operator-68b757fb67-vcwql" Jul 07 18:50:29.175358 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.175281 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3b0a294-e7a3-49fa-a089-92d4bc757e8e-tmp\") pod \"insights-operator-5c847dcfc4-qx6d9\" (UID: \"c3b0a294-e7a3-49fa-a089-92d4bc757e8e\") " pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" Jul 07 18:50:29.175440 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.175419 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/793c0cea-e51b-4fe8-89ba-425492267b6a-trusted-ca\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.175490 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.175419 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3b0a294-e7a3-49fa-a089-92d4bc757e8e-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-qx6d9\" (UID: \"c3b0a294-e7a3-49fa-a089-92d4bc757e8e\") " pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" Jul 07 18:50:29.175542 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.175519 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c3b0a294-e7a3-49fa-a089-92d4bc757e8e-serving-cert\") pod \"insights-operator-5c847dcfc4-qx6d9\" (UID: \"c3b0a294-e7a3-49fa-a089-92d4bc757e8e\") " pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" Jul 07 18:50:29.175629 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.175610 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-metrics-certs\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:29.175693 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.175668 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7a3286ca-3966-48e7-a4d9-9830a7fb1063-config\") pod \"console-operator-68b757fb67-vcwql\" (UID: \"7a3286ca-3966-48e7-a4d9-9830a7fb1063\") " pod="openshift-console-operator/console-operator-68b757fb67-vcwql" Jul 07 18:50:29.175750 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.175734 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7a3286ca-3966-48e7-a4d9-9830a7fb1063-trusted-ca\") pod \"console-operator-68b757fb67-vcwql\" (UID: \"7a3286ca-3966-48e7-a4d9-9830a7fb1063\") " pod="openshift-console-operator/console-operator-68b757fb67-vcwql" Jul 07 18:50:29.175793 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.175767 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ce43d53-3718-49e6-a60e-ce5ca36877ca-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-blnhp\" (UID: \"1ce43d53-3718-49e6-a60e-ce5ca36877ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-blnhp" Jul 07 18:50:29.175835 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.175791 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-certificates\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.175835 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.175807 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9c701b97-bb01-4f1b-af53-8f310ec406ca-config\") pod \"service-ca-operator-5d46d9ffd4-wk9lb\" (UID: \"9c701b97-bb01-4f1b-af53-8f310ec406ca\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk9lb" Jul 07 18:50:29.175914 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.175835 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlf7w\" (UniqueName: \"kubernetes.io/projected/9c701b97-bb01-4f1b-af53-8f310ec406ca-kube-api-access-jlf7w\") pod \"service-ca-operator-5d46d9ffd4-wk9lb\" (UID: \"9c701b97-bb01-4f1b-af53-8f310ec406ca\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk9lb" Jul 07 18:50:29.175956 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.175897 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dppk2\" (UniqueName: \"kubernetes.io/projected/29bdac91-8043-4225-9c92-0d90a7d51003-kube-api-access-dppk2\") pod \"volume-data-source-validator-5d456fdbcd-55mgr\" (UID: \"29bdac91-8043-4225-9c92-0d90a7d51003\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-55mgr" Jul 07 18:50:29.176052 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.176031 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d6822119-b2f7-4a01-9738-66ee52dac067-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-c4ql5\" (UID: \"d6822119-b2f7-4a01-9738-66ee52dac067\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-c4ql5" Jul 07 18:50:29.176106 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.176082 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/793c0cea-e51b-4fe8-89ba-425492267b6a-ca-trust-extracted\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.176157 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.176108 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-bound-sa-token\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.176157 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.176132 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/23e1f849-e113-4a42-bc58-448388e582ce-config-volume\") pod \"dns-default-wvsd8\" (UID: \"23e1f849-e113-4a42-bc58-448388e582ce\") " pod="openshift-dns/dns-default-wvsd8" Jul 07 18:50:29.176253 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.176155 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-service-ca-bundle\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:29.176253 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.176199 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c3b0a294-e7a3-49fa-a089-92d4bc757e8e-snapshots\") pod \"insights-operator-5c847dcfc4-qx6d9\" (UID: \"c3b0a294-e7a3-49fa-a089-92d4bc757e8e\") " pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" Jul 07 18:50:29.176253 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.176224 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2282n\" (UniqueName: \"kubernetes.io/projected/c3b0a294-e7a3-49fa-a089-92d4bc757e8e-kube-api-access-2282n\") pod \"insights-operator-5c847dcfc4-qx6d9\" (UID: \"c3b0a294-e7a3-49fa-a089-92d4bc757e8e\") " pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" Jul 07 18:50:29.176708 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.176688 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7a3286ca-3966-48e7-a4d9-9830a7fb1063-config\") pod \"console-operator-68b757fb67-vcwql\" (UID: \"7a3286ca-3966-48e7-a4d9-9830a7fb1063\") " pod="openshift-console-operator/console-operator-68b757fb67-vcwql" Jul 07 18:50:29.176748 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.176691 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7a3286ca-3966-48e7-a4d9-9830a7fb1063-trusted-ca\") pod \"console-operator-68b757fb67-vcwql\" (UID: \"7a3286ca-3966-48e7-a4d9-9830a7fb1063\") " pod="openshift-console-operator/console-operator-68b757fb67-vcwql" Jul 07 18:50:29.176787 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.176769 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/793c0cea-e51b-4fe8-89ba-425492267b6a-ca-trust-extracted\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.177200 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.177180 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c3b0a294-e7a3-49fa-a089-92d4bc757e8e-snapshots\") pod \"insights-operator-5c847dcfc4-qx6d9\" (UID: \"c3b0a294-e7a3-49fa-a089-92d4bc757e8e\") " pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" Jul 07 18:50:29.178897 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.178876 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7a3286ca-3966-48e7-a4d9-9830a7fb1063-serving-cert\") pod \"console-operator-68b757fb67-vcwql\" (UID: \"7a3286ca-3966-48e7-a4d9-9830a7fb1063\") " pod="openshift-console-operator/console-operator-68b757fb67-vcwql" Jul 07 18:50:29.179071 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.179047 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c3b0a294-e7a3-49fa-a089-92d4bc757e8e-serving-cert\") pod \"insights-operator-5c847dcfc4-qx6d9\" (UID: \"c3b0a294-e7a3-49fa-a089-92d4bc757e8e\") " pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" Jul 07 18:50:29.179247 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.179231 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/793c0cea-e51b-4fe8-89ba-425492267b6a-installation-pull-secrets\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.179538 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.179521 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/793c0cea-e51b-4fe8-89ba-425492267b6a-image-registry-private-configuration\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.186769 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.186706 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrjm7\" (UniqueName: \"kubernetes.io/projected/7a3286ca-3966-48e7-a4d9-9830a7fb1063-kube-api-access-jrjm7\") pod \"console-operator-68b757fb67-vcwql\" (UID: \"7a3286ca-3966-48e7-a4d9-9830a7fb1063\") " pod="openshift-console-operator/console-operator-68b757fb67-vcwql" Jul 07 18:50:29.187497 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.187477 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfkrx\" (UniqueName: \"kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-kube-api-access-gfkrx\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.189288 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.189266 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-bound-sa-token\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.190302 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.190281 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2282n\" (UniqueName: \"kubernetes.io/projected/c3b0a294-e7a3-49fa-a089-92d4bc757e8e-kube-api-access-2282n\") pod \"insights-operator-5c847dcfc4-qx6d9\" (UID: \"c3b0a294-e7a3-49fa-a089-92d4bc757e8e\") " pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" Jul 07 18:50:29.277452 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.277421 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ce43d53-3718-49e6-a60e-ce5ca36877ca-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-blnhp\" (UID: \"1ce43d53-3718-49e6-a60e-ce5ca36877ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-blnhp" Jul 07 18:50:29.277639 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.277463 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9c701b97-bb01-4f1b-af53-8f310ec406ca-config\") pod \"service-ca-operator-5d46d9ffd4-wk9lb\" (UID: \"9c701b97-bb01-4f1b-af53-8f310ec406ca\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk9lb" Jul 07 18:50:29.277639 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.277489 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlf7w\" (UniqueName: \"kubernetes.io/projected/9c701b97-bb01-4f1b-af53-8f310ec406ca-kube-api-access-jlf7w\") pod \"service-ca-operator-5d46d9ffd4-wk9lb\" (UID: \"9c701b97-bb01-4f1b-af53-8f310ec406ca\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk9lb" Jul 07 18:50:29.277639 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.277516 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dppk2\" (UniqueName: \"kubernetes.io/projected/29bdac91-8043-4225-9c92-0d90a7d51003-kube-api-access-dppk2\") pod \"volume-data-source-validator-5d456fdbcd-55mgr\" (UID: \"29bdac91-8043-4225-9c92-0d90a7d51003\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-55mgr" Jul 07 18:50:29.277639 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.277546 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d6822119-b2f7-4a01-9738-66ee52dac067-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-c4ql5\" (UID: \"d6822119-b2f7-4a01-9738-66ee52dac067\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-c4ql5" Jul 07 18:50:29.277639 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.277595 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/23e1f849-e113-4a42-bc58-448388e582ce-config-volume\") pod \"dns-default-wvsd8\" (UID: \"23e1f849-e113-4a42-bc58-448388e582ce\") " pod="openshift-dns/dns-default-wvsd8" Jul 07 18:50:29.277639 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.277620 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-service-ca-bundle\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:29.277984 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.277673 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/10350357-c651-4b89-8497-7e16a1b6c701-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-b5nnl\" (UID: \"10350357-c651-4b89-8497-7e16a1b6c701\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl" Jul 07 18:50:29.277984 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.277701 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kchhf\" (UniqueName: \"kubernetes.io/projected/76ee52c5-2d6b-4c13-b50f-312157fc165f-kube-api-access-kchhf\") pod \"ingress-canary-9s59k\" (UID: \"76ee52c5-2d6b-4c13-b50f-312157fc165f\") " pod="openshift-ingress-canary/ingress-canary-9s59k" Jul 07 18:50:29.277984 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.277728 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf822\" (UniqueName: \"kubernetes.io/projected/1ce43d53-3718-49e6-a60e-ce5ca36877ca-kube-api-access-wf822\") pod \"kube-storage-version-migrator-operator-846bf668f7-blnhp\" (UID: \"1ce43d53-3718-49e6-a60e-ce5ca36877ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-blnhp" Jul 07 18:50:29.277984 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.277768 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qd4j\" (UniqueName: \"kubernetes.io/projected/23e1f849-e113-4a42-bc58-448388e582ce-kube-api-access-5qd4j\") pod \"dns-default-wvsd8\" (UID: \"23e1f849-e113-4a42-bc58-448388e582ce\") " pod="openshift-dns/dns-default-wvsd8" Jul 07 18:50:29.277984 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.277817 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10350357-c651-4b89-8497-7e16a1b6c701-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-b5nnl\" (UID: \"10350357-c651-4b89-8497-7e16a1b6c701\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl" Jul 07 18:50:29.277984 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.277842 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ce43d53-3718-49e6-a60e-ce5ca36877ca-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-blnhp\" (UID: \"1ce43d53-3718-49e6-a60e-ce5ca36877ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-blnhp" Jul 07 18:50:29.277984 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.277864 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-service-ca-bundle podName:2003b9f1-daee-437f-8f44-9ce42ae8f6bd nodeName:}" failed. No retries permitted until 2026-07-07 18:50:29.777844849 +0000 UTC m=+37.197205069 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-service-ca-bundle") pod "router-default-7dd847dbd5-mq8mz" (UID: "2003b9f1-daee-437f-8f44-9ce42ae8f6bd") : configmap references non-existent config key: service-ca.crt Jul 07 18:50:29.277984 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.277907 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-default-certificate\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:29.277984 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.277936 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23e1f849-e113-4a42-bc58-448388e582ce-metrics-tls\") pod \"dns-default-wvsd8\" (UID: \"23e1f849-e113-4a42-bc58-448388e582ce\") " pod="openshift-dns/dns-default-wvsd8" Jul 07 18:50:29.278408 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.278005 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-stats-auth\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:29.278408 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.278042 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76ee52c5-2d6b-4c13-b50f-312157fc165f-cert\") pod \"ingress-canary-9s59k\" (UID: \"76ee52c5-2d6b-4c13-b50f-312157fc165f\") " pod="openshift-ingress-canary/ingress-canary-9s59k" Jul 07 18:50:29.278408 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.278062 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ce43d53-3718-49e6-a60e-ce5ca36877ca-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-blnhp\" (UID: \"1ce43d53-3718-49e6-a60e-ce5ca36877ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-blnhp" Jul 07 18:50:29.278408 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.278074 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvzsd\" (UniqueName: \"kubernetes.io/projected/d6822119-b2f7-4a01-9738-66ee52dac067-kube-api-access-kvzsd\") pod \"cluster-samples-operator-56fcf86bf-c4ql5\" (UID: \"d6822119-b2f7-4a01-9738-66ee52dac067\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-c4ql5" Jul 07 18:50:29.278408 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.278122 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9c701b97-bb01-4f1b-af53-8f310ec406ca-config\") pod \"service-ca-operator-5d46d9ffd4-wk9lb\" (UID: \"9c701b97-bb01-4f1b-af53-8f310ec406ca\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk9lb" Jul 07 18:50:29.278408 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.278131 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23e1f849-e113-4a42-bc58-448388e582ce-tmp-dir\") pod \"dns-default-wvsd8\" (UID: \"23e1f849-e113-4a42-bc58-448388e582ce\") " pod="openshift-dns/dns-default-wvsd8" Jul 07 18:50:29.278408 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.278166 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:50:29.278408 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.278302 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2k6l\" (UniqueName: \"kubernetes.io/projected/10350357-c651-4b89-8497-7e16a1b6c701-kube-api-access-r2k6l\") pod \"cluster-monitoring-operator-64786d9cbb-b5nnl\" (UID: \"10350357-c651-4b89-8497-7e16a1b6c701\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl" Jul 07 18:50:29.278408 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.278325 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmb2k\" (UniqueName: \"kubernetes.io/projected/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-kube-api-access-xmb2k\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:29.278408 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.278349 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz6bw\" (UniqueName: \"kubernetes.io/projected/f868dc34-c7da-4e41-abd3-8ad56bae2bc6-kube-api-access-jz6bw\") pod \"network-check-source-989f8b8dc-mq4f7\" (UID: \"f868dc34-c7da-4e41-abd3-8ad56bae2bc6\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mq4f7" Jul 07 18:50:29.278408 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.278366 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9c701b97-bb01-4f1b-af53-8f310ec406ca-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-wk9lb\" (UID: \"9c701b97-bb01-4f1b-af53-8f310ec406ca\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk9lb" Jul 07 18:50:29.278408 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.278390 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/10350357-c651-4b89-8497-7e16a1b6c701-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-b5nnl\" (UID: \"10350357-c651-4b89-8497-7e16a1b6c701\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl" Jul 07 18:50:29.278408 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.278413 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-metrics-certs\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:29.279184 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.278436 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:29.279184 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.278473 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10350357-c651-4b89-8497-7e16a1b6c701-cluster-monitoring-operator-tls podName:10350357-c651-4b89-8497-7e16a1b6c701 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:29.778461975 +0000 UTC m=+37.197822177 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/10350357-c651-4b89-8497-7e16a1b6c701-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-b5nnl" (UID: "10350357-c651-4b89-8497-7e16a1b6c701") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:29.279184 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.278629 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 18:50:29.279184 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.278686 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6822119-b2f7-4a01-9738-66ee52dac067-samples-operator-tls podName:d6822119-b2f7-4a01-9738-66ee52dac067 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:29.778672543 +0000 UTC m=+37.198032774 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d6822119-b2f7-4a01-9738-66ee52dac067-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-c4ql5" (UID: "d6822119-b2f7-4a01-9738-66ee52dac067") : secret "samples-operator-tls" not found Jul 07 18:50:29.279184 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.278914 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23e1f849-e113-4a42-bc58-448388e582ce-tmp-dir\") pod \"dns-default-wvsd8\" (UID: \"23e1f849-e113-4a42-bc58-448388e582ce\") " pod="openshift-dns/dns-default-wvsd8" Jul 07 18:50:29.279184 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.279051 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/23e1f849-e113-4a42-bc58-448388e582ce-config-volume\") pod \"dns-default-wvsd8\" (UID: \"23e1f849-e113-4a42-bc58-448388e582ce\") " pod="openshift-dns/dns-default-wvsd8" Jul 07 18:50:29.279184 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.279059 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23e1f849-e113-4a42-bc58-448388e582ce-metrics-tls podName:23e1f849-e113-4a42-bc58-448388e582ce nodeName:}" failed. No retries permitted until 2026-07-07 18:50:29.779011803 +0000 UTC m=+37.198374282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/23e1f849-e113-4a42-bc58-448388e582ce-metrics-tls") pod "dns-default-wvsd8" (UID: "23e1f849-e113-4a42-bc58-448388e582ce") : secret "dns-default-metrics-tls" not found Jul 07 18:50:29.279184 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.279122 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 18:50:29.279184 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.279162 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-metrics-certs podName:2003b9f1-daee-437f-8f44-9ce42ae8f6bd nodeName:}" failed. No retries permitted until 2026-07-07 18:50:29.779149226 +0000 UTC m=+37.198509431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-metrics-certs") pod "router-default-7dd847dbd5-mq8mz" (UID: "2003b9f1-daee-437f-8f44-9ce42ae8f6bd") : secret "router-metrics-certs-default" not found Jul 07 18:50:29.280861 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.280840 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-stats-auth\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:29.280979 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.280961 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-default-certificate\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:29.281260 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.281236 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ce43d53-3718-49e6-a60e-ce5ca36877ca-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-blnhp\" (UID: \"1ce43d53-3718-49e6-a60e-ce5ca36877ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-blnhp" Jul 07 18:50:29.281486 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.281432 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9c701b97-bb01-4f1b-af53-8f310ec406ca-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-wk9lb\" (UID: \"9c701b97-bb01-4f1b-af53-8f310ec406ca\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk9lb" Jul 07 18:50:29.290668 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.290613 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf822\" (UniqueName: \"kubernetes.io/projected/1ce43d53-3718-49e6-a60e-ce5ca36877ca-kube-api-access-wf822\") pod \"kube-storage-version-migrator-operator-846bf668f7-blnhp\" (UID: \"1ce43d53-3718-49e6-a60e-ce5ca36877ca\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-blnhp" Jul 07 18:50:29.290975 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.290947 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qd4j\" (UniqueName: \"kubernetes.io/projected/23e1f849-e113-4a42-bc58-448388e582ce-kube-api-access-5qd4j\") pod \"dns-default-wvsd8\" (UID: \"23e1f849-e113-4a42-bc58-448388e582ce\") " pod="openshift-dns/dns-default-wvsd8" Jul 07 18:50:29.291499 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.291480 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" Jul 07 18:50:29.291615 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.291478 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dppk2\" (UniqueName: \"kubernetes.io/projected/29bdac91-8043-4225-9c92-0d90a7d51003-kube-api-access-dppk2\") pod \"volume-data-source-validator-5d456fdbcd-55mgr\" (UID: \"29bdac91-8043-4225-9c92-0d90a7d51003\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-55mgr" Jul 07 18:50:29.291858 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.291840 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvzsd\" (UniqueName: \"kubernetes.io/projected/d6822119-b2f7-4a01-9738-66ee52dac067-kube-api-access-kvzsd\") pod \"cluster-samples-operator-56fcf86bf-c4ql5\" (UID: \"d6822119-b2f7-4a01-9738-66ee52dac067\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-c4ql5" Jul 07 18:50:29.292611 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.292561 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlf7w\" (UniqueName: \"kubernetes.io/projected/9c701b97-bb01-4f1b-af53-8f310ec406ca-kube-api-access-jlf7w\") pod \"service-ca-operator-5d46d9ffd4-wk9lb\" (UID: \"9c701b97-bb01-4f1b-af53-8f310ec406ca\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk9lb" Jul 07 18:50:29.293235 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.293213 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmb2k\" (UniqueName: \"kubernetes.io/projected/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-kube-api-access-xmb2k\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:29.293634 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.293611 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2k6l\" (UniqueName: \"kubernetes.io/projected/10350357-c651-4b89-8497-7e16a1b6c701-kube-api-access-r2k6l\") pod \"cluster-monitoring-operator-64786d9cbb-b5nnl\" (UID: \"10350357-c651-4b89-8497-7e16a1b6c701\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl" Jul 07 18:50:29.293969 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.293951 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz6bw\" (UniqueName: \"kubernetes.io/projected/f868dc34-c7da-4e41-abd3-8ad56bae2bc6-kube-api-access-jz6bw\") pod \"network-check-source-989f8b8dc-mq4f7\" (UID: \"f868dc34-c7da-4e41-abd3-8ad56bae2bc6\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mq4f7" Jul 07 18:50:29.304018 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.303997 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-vcwql" Jul 07 18:50:29.321561 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.321517 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk9lb" Jul 07 18:50:29.328369 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.328356 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-55mgr" Jul 07 18:50:29.340583 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.340548 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-blnhp" Jul 07 18:50:29.358644 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.358624 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mq4f7" Jul 07 18:50:29.379648 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.379626 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kchhf\" (UniqueName: \"kubernetes.io/projected/76ee52c5-2d6b-4c13-b50f-312157fc165f-kube-api-access-kchhf\") pod \"ingress-canary-9s59k\" (UID: \"76ee52c5-2d6b-4c13-b50f-312157fc165f\") " pod="openshift-ingress-canary/ingress-canary-9s59k" Jul 07 18:50:29.379738 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.379688 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76ee52c5-2d6b-4c13-b50f-312157fc165f-cert\") pod \"ingress-canary-9s59k\" (UID: \"76ee52c5-2d6b-4c13-b50f-312157fc165f\") " pod="openshift-ingress-canary/ingress-canary-9s59k" Jul 07 18:50:29.382102 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.382078 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/76ee52c5-2d6b-4c13-b50f-312157fc165f-cert\") pod \"ingress-canary-9s59k\" (UID: \"76ee52c5-2d6b-4c13-b50f-312157fc165f\") " pod="openshift-ingress-canary/ingress-canary-9s59k" Jul 07 18:50:29.389422 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.389406 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kchhf\" (UniqueName: \"kubernetes.io/projected/76ee52c5-2d6b-4c13-b50f-312157fc165f-kube-api-access-kchhf\") pod \"ingress-canary-9s59k\" (UID: \"76ee52c5-2d6b-4c13-b50f-312157fc165f\") " pod="openshift-ingress-canary/ingress-canary-9s59k" Jul 07 18:50:29.452770 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.452726 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9s59k" Jul 07 18:50:29.682273 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.682236 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:29.682428 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.682413 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 18:50:29.682471 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.682432 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c64b8fbfb-nfsfs: secret "image-registry-tls" not found Jul 07 18:50:29.682511 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.682481 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls podName:793c0cea-e51b-4fe8-89ba-425492267b6a nodeName:}" failed. No retries permitted until 2026-07-07 18:50:30.682466231 +0000 UTC m=+38.101826434 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls") pod "image-registry-6c64b8fbfb-nfsfs" (UID: "793c0cea-e51b-4fe8-89ba-425492267b6a") : secret "image-registry-tls" not found Jul 07 18:50:29.783305 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.783224 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23e1f849-e113-4a42-bc58-448388e582ce-metrics-tls\") pod \"dns-default-wvsd8\" (UID: \"23e1f849-e113-4a42-bc58-448388e582ce\") " pod="openshift-dns/dns-default-wvsd8" Jul 07 18:50:29.783453 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.783317 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-metrics-certs\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:29.783453 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.783356 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d6822119-b2f7-4a01-9738-66ee52dac067-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-c4ql5\" (UID: \"d6822119-b2f7-4a01-9738-66ee52dac067\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-c4ql5" Jul 07 18:50:29.783453 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.783377 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:50:29.783453 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.783389 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-service-ca-bundle\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:29.783453 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.783443 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23e1f849-e113-4a42-bc58-448388e582ce-metrics-tls podName:23e1f849-e113-4a42-bc58-448388e582ce nodeName:}" failed. No retries permitted until 2026-07-07 18:50:30.783425361 +0000 UTC m=+38.202785588 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/23e1f849-e113-4a42-bc58-448388e582ce-metrics-tls") pod "dns-default-wvsd8" (UID: "23e1f849-e113-4a42-bc58-448388e582ce") : secret "dns-default-metrics-tls" not found Jul 07 18:50:29.783641 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:29.783483 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10350357-c651-4b89-8497-7e16a1b6c701-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-b5nnl\" (UID: \"10350357-c651-4b89-8497-7e16a1b6c701\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl" Jul 07 18:50:29.783641 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.783521 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-service-ca-bundle podName:2003b9f1-daee-437f-8f44-9ce42ae8f6bd nodeName:}" failed. No retries permitted until 2026-07-07 18:50:30.783509756 +0000 UTC m=+38.202869958 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-service-ca-bundle") pod "router-default-7dd847dbd5-mq8mz" (UID: "2003b9f1-daee-437f-8f44-9ce42ae8f6bd") : configmap references non-existent config key: service-ca.crt Jul 07 18:50:29.783641 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.783554 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:29.783641 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.783593 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10350357-c651-4b89-8497-7e16a1b6c701-cluster-monitoring-operator-tls podName:10350357-c651-4b89-8497-7e16a1b6c701 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:30.783585685 +0000 UTC m=+38.202945899 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/10350357-c651-4b89-8497-7e16a1b6c701-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-b5nnl" (UID: "10350357-c651-4b89-8497-7e16a1b6c701") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:29.783641 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.783592 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 18:50:29.783641 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.783618 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6822119-b2f7-4a01-9738-66ee52dac067-samples-operator-tls podName:d6822119-b2f7-4a01-9738-66ee52dac067 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:30.783611036 +0000 UTC m=+38.202971239 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d6822119-b2f7-4a01-9738-66ee52dac067-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-c4ql5" (UID: "d6822119-b2f7-4a01-9738-66ee52dac067") : secret "samples-operator-tls" not found Jul 07 18:50:29.783641 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.783623 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 18:50:29.783641 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:29.783641 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-metrics-certs podName:2003b9f1-daee-437f-8f44-9ce42ae8f6bd nodeName:}" failed. No retries permitted until 2026-07-07 18:50:30.783635658 +0000 UTC m=+38.202995860 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-metrics-certs") pod "router-default-7dd847dbd5-mq8mz" (UID: "2003b9f1-daee-437f-8f44-9ce42ae8f6bd") : secret "router-metrics-certs-default" not found Jul 07 18:50:30.695486 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:30.695220 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:30.696288 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:30.695636 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 18:50:30.696288 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:30.695654 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c64b8fbfb-nfsfs: secret "image-registry-tls" not found Jul 07 18:50:30.696288 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:30.695713 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls podName:793c0cea-e51b-4fe8-89ba-425492267b6a nodeName:}" failed. No retries permitted until 2026-07-07 18:50:32.695695219 +0000 UTC m=+40.115055444 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls") pod "image-registry-6c64b8fbfb-nfsfs" (UID: "793c0cea-e51b-4fe8-89ba-425492267b6a") : secret "image-registry-tls" not found Jul 07 18:50:30.796204 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:30.796169 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d6822119-b2f7-4a01-9738-66ee52dac067-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-c4ql5\" (UID: \"d6822119-b2f7-4a01-9738-66ee52dac067\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-c4ql5" Jul 07 18:50:30.796352 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:30.796224 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-service-ca-bundle\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:30.796352 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:30.796314 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10350357-c651-4b89-8497-7e16a1b6c701-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-b5nnl\" (UID: \"10350357-c651-4b89-8497-7e16a1b6c701\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl" Jul 07 18:50:30.796352 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:30.796351 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23e1f849-e113-4a42-bc58-448388e582ce-metrics-tls\") pod \"dns-default-wvsd8\" (UID: \"23e1f849-e113-4a42-bc58-448388e582ce\") " pod="openshift-dns/dns-default-wvsd8" Jul 07 18:50:30.796519 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:30.796424 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-metrics-certs\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:30.796604 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:30.796551 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 18:50:30.796661 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:30.796632 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-metrics-certs podName:2003b9f1-daee-437f-8f44-9ce42ae8f6bd nodeName:}" failed. No retries permitted until 2026-07-07 18:50:32.796612944 +0000 UTC m=+40.215973158 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-metrics-certs") pod "router-default-7dd847dbd5-mq8mz" (UID: "2003b9f1-daee-437f-8f44-9ce42ae8f6bd") : secret "router-metrics-certs-default" not found Jul 07 18:50:30.797041 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:30.797021 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 18:50:30.797144 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:30.797079 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6822119-b2f7-4a01-9738-66ee52dac067-samples-operator-tls podName:d6822119-b2f7-4a01-9738-66ee52dac067 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:32.797062656 +0000 UTC m=+40.216422865 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d6822119-b2f7-4a01-9738-66ee52dac067-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-c4ql5" (UID: "d6822119-b2f7-4a01-9738-66ee52dac067") : secret "samples-operator-tls" not found Jul 07 18:50:30.797376 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:30.797360 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-service-ca-bundle podName:2003b9f1-daee-437f-8f44-9ce42ae8f6bd nodeName:}" failed. No retries permitted until 2026-07-07 18:50:32.797349306 +0000 UTC m=+40.216709508 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-service-ca-bundle") pod "router-default-7dd847dbd5-mq8mz" (UID: "2003b9f1-daee-437f-8f44-9ce42ae8f6bd") : configmap references non-existent config key: service-ca.crt Jul 07 18:50:30.797514 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:30.797424 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:30.797514 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:30.797461 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10350357-c651-4b89-8497-7e16a1b6c701-cluster-monitoring-operator-tls podName:10350357-c651-4b89-8497-7e16a1b6c701 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:32.797448047 +0000 UTC m=+40.216808255 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/10350357-c651-4b89-8497-7e16a1b6c701-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-b5nnl" (UID: "10350357-c651-4b89-8497-7e16a1b6c701") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:30.797514 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:30.797510 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:50:30.797691 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:30.797538 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23e1f849-e113-4a42-bc58-448388e582ce-metrics-tls podName:23e1f849-e113-4a42-bc58-448388e582ce nodeName:}" failed. No retries permitted until 2026-07-07 18:50:32.79752815 +0000 UTC m=+40.216888352 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/23e1f849-e113-4a42-bc58-448388e582ce-metrics-tls") pod "dns-default-wvsd8" (UID: "23e1f849-e113-4a42-bc58-448388e582ce") : secret "dns-default-metrics-tls" not found Jul 07 18:50:30.810048 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:30.810022 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9s59k"] Jul 07 18:50:30.813048 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:50:30.812257 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76ee52c5_2d6b_4c13_b50f_312157fc165f.slice/crio-75d7c7e3a20018407caee8ff6e6dba89c22a4b266a3e8ce637adb4b610e4579d WatchSource:0}: Error finding container 75d7c7e3a20018407caee8ff6e6dba89c22a4b266a3e8ce637adb4b610e4579d: Status 404 returned error can't find the container with id 75d7c7e3a20018407caee8ff6e6dba89c22a4b266a3e8ce637adb4b610e4579d Jul 07 18:50:30.813385 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:30.813357 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-qx6d9"] Jul 07 18:50:30.817964 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:30.816727 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-55mgr"] Jul 07 18:50:30.818918 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:30.818598 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-blnhp"] Jul 07 18:50:30.821795 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:50:30.821771 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29bdac91_8043_4225_9c92_0d90a7d51003.slice/crio-754fe35c1869521ece1b64e62efbd5c6e437701f3e59bc2273a8e768dc3e353d WatchSource:0}: Error finding container 754fe35c1869521ece1b64e62efbd5c6e437701f3e59bc2273a8e768dc3e353d: Status 404 returned error can't find the container with id 754fe35c1869521ece1b64e62efbd5c6e437701f3e59bc2273a8e768dc3e353d Jul 07 18:50:30.822438 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:50:30.822414 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ce43d53_3718_49e6_a60e_ce5ca36877ca.slice/crio-6e74e8b82b7fedb62ee58a663c5561b976d8709a918df41aa71c6864769bbcc9 WatchSource:0}: Error finding container 6e74e8b82b7fedb62ee58a663c5561b976d8709a918df41aa71c6864769bbcc9: Status 404 returned error can't find the container with id 6e74e8b82b7fedb62ee58a663c5561b976d8709a918df41aa71c6864769bbcc9 Jul 07 18:50:30.826175 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:30.826158 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk9lb"] Jul 07 18:50:30.829128 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:50:30.829098 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c701b97_bb01_4f1b_af53_8f310ec406ca.slice/crio-bb4797eb500a4ae9709fa5467376eba4556eaea4a1aaec0f02cdc171e8a4c6a2 WatchSource:0}: Error finding container bb4797eb500a4ae9709fa5467376eba4556eaea4a1aaec0f02cdc171e8a4c6a2: Status 404 returned error can't find the container with id bb4797eb500a4ae9709fa5467376eba4556eaea4a1aaec0f02cdc171e8a4c6a2 Jul 07 18:50:30.833822 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:30.833800 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-mq4f7"] Jul 07 18:50:30.836195 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:30.836175 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-vcwql"] Jul 07 18:50:30.841178 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:50:30.841074 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a3286ca_3966_48e7_a4d9_9830a7fb1063.slice/crio-4b6aff0cb4ccea247ad6c6b5ced7f6af6b5fe338936996817067f15973d48841 WatchSource:0}: Error finding container 4b6aff0cb4ccea247ad6c6b5ced7f6af6b5fe338936996817067f15973d48841: Status 404 returned error can't find the container with id 4b6aff0cb4ccea247ad6c6b5ced7f6af6b5fe338936996817067f15973d48841 Jul 07 18:50:30.841975 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:50:30.841936 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf868dc34_c7da_4e41_abd3_8ad56bae2bc6.slice/crio-5dae675f05efaa4768222f79a767a1a4e38f31d3bd9229227d15ffe41fb43858 WatchSource:0}: Error finding container 5dae675f05efaa4768222f79a767a1a4e38f31d3bd9229227d15ffe41fb43858: Status 404 returned error can't find the container with id 5dae675f05efaa4768222f79a767a1a4e38f31d3bd9229227d15ffe41fb43858 Jul 07 18:50:31.253698 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:31.253615 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-vcwql" event={"ID":"7a3286ca-3966-48e7-a4d9-9830a7fb1063","Type":"ContainerStarted","Data":"4b6aff0cb4ccea247ad6c6b5ced7f6af6b5fe338936996817067f15973d48841"} Jul 07 18:50:31.254846 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:31.254808 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-55mgr" event={"ID":"29bdac91-8043-4225-9c92-0d90a7d51003","Type":"ContainerStarted","Data":"754fe35c1869521ece1b64e62efbd5c6e437701f3e59bc2273a8e768dc3e353d"} Jul 07 18:50:31.256013 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:31.255985 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk9lb" event={"ID":"9c701b97-bb01-4f1b-af53-8f310ec406ca","Type":"ContainerStarted","Data":"bb4797eb500a4ae9709fa5467376eba4556eaea4a1aaec0f02cdc171e8a4c6a2"} Jul 07 18:50:31.257191 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:31.257172 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-blnhp" event={"ID":"1ce43d53-3718-49e6-a60e-ce5ca36877ca","Type":"ContainerStarted","Data":"6e74e8b82b7fedb62ee58a663c5561b976d8709a918df41aa71c6864769bbcc9"} Jul 07 18:50:31.258515 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:31.258494 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" event={"ID":"c3b0a294-e7a3-49fa-a089-92d4bc757e8e","Type":"ContainerStarted","Data":"acb2b65946087d25fa43314e96cff1fa51e507520a09e939678d5241a99c57f7"} Jul 07 18:50:31.261809 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:31.261785 2564 generic.go:356] "Generic (PLEG): container finished" podID="09b26336-e7e1-40b3-b55c-f2640139c685" containerID="1bfb72400810712609e08d33abbabb0a74b7d1f316a25e8eaf741079cefbae79" exitCode=0 Jul 07 18:50:31.261890 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:31.261865 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8pkcb" event={"ID":"09b26336-e7e1-40b3-b55c-f2640139c685","Type":"ContainerDied","Data":"1bfb72400810712609e08d33abbabb0a74b7d1f316a25e8eaf741079cefbae79"} Jul 07 18:50:31.263058 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:31.263025 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mq4f7" event={"ID":"f868dc34-c7da-4e41-abd3-8ad56bae2bc6","Type":"ContainerStarted","Data":"5dae675f05efaa4768222f79a767a1a4e38f31d3bd9229227d15ffe41fb43858"} Jul 07 18:50:31.264469 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:31.264446 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9s59k" event={"ID":"76ee52c5-2d6b-4c13-b50f-312157fc165f","Type":"ContainerStarted","Data":"75d7c7e3a20018407caee8ff6e6dba89c22a4b266a3e8ce637adb4b610e4579d"} Jul 07 18:50:32.280188 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:32.280148 2564 generic.go:356] "Generic (PLEG): container finished" podID="09b26336-e7e1-40b3-b55c-f2640139c685" containerID="f8f508a385c92180f43a7581092c29f90cc4e98950df21f63ff1da435daa5745" exitCode=0 Jul 07 18:50:32.280697 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:32.280230 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8pkcb" event={"ID":"09b26336-e7e1-40b3-b55c-f2640139c685","Type":"ContainerDied","Data":"f8f508a385c92180f43a7581092c29f90cc4e98950df21f63ff1da435daa5745"} Jul 07 18:50:32.717086 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:32.717048 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:32.717297 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:32.717228 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 18:50:32.717297 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:32.717245 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c64b8fbfb-nfsfs: secret "image-registry-tls" not found Jul 07 18:50:32.717297 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:32.717298 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls podName:793c0cea-e51b-4fe8-89ba-425492267b6a nodeName:}" failed. No retries permitted until 2026-07-07 18:50:36.717282472 +0000 UTC m=+44.136642688 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls") pod "image-registry-6c64b8fbfb-nfsfs" (UID: "793c0cea-e51b-4fe8-89ba-425492267b6a") : secret "image-registry-tls" not found Jul 07 18:50:32.818401 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:32.818316 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10350357-c651-4b89-8497-7e16a1b6c701-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-b5nnl\" (UID: \"10350357-c651-4b89-8497-7e16a1b6c701\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl" Jul 07 18:50:32.818401 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:32.818371 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23e1f849-e113-4a42-bc58-448388e582ce-metrics-tls\") pod \"dns-default-wvsd8\" (UID: \"23e1f849-e113-4a42-bc58-448388e582ce\") " pod="openshift-dns/dns-default-wvsd8" Jul 07 18:50:32.818673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:32.818437 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-metrics-certs\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:32.818673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:32.818471 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d6822119-b2f7-4a01-9738-66ee52dac067-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-c4ql5\" (UID: \"d6822119-b2f7-4a01-9738-66ee52dac067\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-c4ql5" Jul 07 18:50:32.818673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:32.818500 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-service-ca-bundle\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:32.818865 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:32.818690 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-service-ca-bundle podName:2003b9f1-daee-437f-8f44-9ce42ae8f6bd nodeName:}" failed. No retries permitted until 2026-07-07 18:50:36.818670249 +0000 UTC m=+44.238030465 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-service-ca-bundle") pod "router-default-7dd847dbd5-mq8mz" (UID: "2003b9f1-daee-437f-8f44-9ce42ae8f6bd") : configmap references non-existent config key: service-ca.crt Jul 07 18:50:32.819110 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:32.819090 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:32.819204 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:32.819145 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10350357-c651-4b89-8497-7e16a1b6c701-cluster-monitoring-operator-tls podName:10350357-c651-4b89-8497-7e16a1b6c701 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:36.819128807 +0000 UTC m=+44.238489028 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/10350357-c651-4b89-8497-7e16a1b6c701-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-b5nnl" (UID: "10350357-c651-4b89-8497-7e16a1b6c701") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:32.819271 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:32.819208 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:50:32.819271 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:32.819241 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23e1f849-e113-4a42-bc58-448388e582ce-metrics-tls podName:23e1f849-e113-4a42-bc58-448388e582ce nodeName:}" failed. No retries permitted until 2026-07-07 18:50:36.819229658 +0000 UTC m=+44.238589860 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/23e1f849-e113-4a42-bc58-448388e582ce-metrics-tls") pod "dns-default-wvsd8" (UID: "23e1f849-e113-4a42-bc58-448388e582ce") : secret "dns-default-metrics-tls" not found Jul 07 18:50:32.819373 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:32.819296 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 18:50:32.819373 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:32.819327 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-metrics-certs podName:2003b9f1-daee-437f-8f44-9ce42ae8f6bd nodeName:}" failed. No retries permitted until 2026-07-07 18:50:36.819317498 +0000 UTC m=+44.238677703 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-metrics-certs") pod "router-default-7dd847dbd5-mq8mz" (UID: "2003b9f1-daee-437f-8f44-9ce42ae8f6bd") : secret "router-metrics-certs-default" not found Jul 07 18:50:32.819507 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:32.819380 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 18:50:32.819507 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:32.819412 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6822119-b2f7-4a01-9738-66ee52dac067-samples-operator-tls podName:d6822119-b2f7-4a01-9738-66ee52dac067 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:36.819401697 +0000 UTC m=+44.238761913 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d6822119-b2f7-4a01-9738-66ee52dac067-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-c4ql5" (UID: "d6822119-b2f7-4a01-9738-66ee52dac067") : secret "samples-operator-tls" not found Jul 07 18:50:33.290597 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:33.289771 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8pkcb" event={"ID":"09b26336-e7e1-40b3-b55c-f2640139c685","Type":"ContainerStarted","Data":"d7ace0920b5ad49c852f71fbbedda3daab69d48adb083267b9b2733375b114f8"} Jul 07 18:50:33.315487 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:33.314782 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8pkcb" podStartSLOduration=8.567719961 podStartE2EDuration="40.314760986s" totalImagesPullingTime="31.747041025s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:53 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:33.313790849 +0000 UTC m=+40.733151073" watchObservedRunningTime="2026-07-07 18:50:33.314760986 +0000 UTC m=+40.734121212" Jul 07 18:50:36.754540 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:36.754490 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:36.754915 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:36.754656 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 18:50:36.754915 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:36.754678 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c64b8fbfb-nfsfs: secret "image-registry-tls" not found Jul 07 18:50:36.754915 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:36.754737 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls podName:793c0cea-e51b-4fe8-89ba-425492267b6a nodeName:}" failed. No retries permitted until 2026-07-07 18:50:44.754722517 +0000 UTC m=+52.174082720 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls") pod "image-registry-6c64b8fbfb-nfsfs" (UID: "793c0cea-e51b-4fe8-89ba-425492267b6a") : secret "image-registry-tls" not found Jul 07 18:50:36.855936 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:36.855897 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10350357-c651-4b89-8497-7e16a1b6c701-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-b5nnl\" (UID: \"10350357-c651-4b89-8497-7e16a1b6c701\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl" Jul 07 18:50:36.856100 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:36.855951 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23e1f849-e113-4a42-bc58-448388e582ce-metrics-tls\") pod \"dns-default-wvsd8\" (UID: \"23e1f849-e113-4a42-bc58-448388e582ce\") " pod="openshift-dns/dns-default-wvsd8" Jul 07 18:50:36.856100 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:36.856052 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:50:36.856100 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:36.856062 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:36.856229 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:36.856118 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23e1f849-e113-4a42-bc58-448388e582ce-metrics-tls podName:23e1f849-e113-4a42-bc58-448388e582ce nodeName:}" failed. No retries permitted until 2026-07-07 18:50:44.8561004 +0000 UTC m=+52.275460628 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/23e1f849-e113-4a42-bc58-448388e582ce-metrics-tls") pod "dns-default-wvsd8" (UID: "23e1f849-e113-4a42-bc58-448388e582ce") : secret "dns-default-metrics-tls" not found Jul 07 18:50:36.856229 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:36.856187 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10350357-c651-4b89-8497-7e16a1b6c701-cluster-monitoring-operator-tls podName:10350357-c651-4b89-8497-7e16a1b6c701 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:44.856167463 +0000 UTC m=+52.275527669 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/10350357-c651-4b89-8497-7e16a1b6c701-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-b5nnl" (UID: "10350357-c651-4b89-8497-7e16a1b6c701") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:36.856301 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:36.856242 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-metrics-certs\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:36.856301 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:36.856271 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d6822119-b2f7-4a01-9738-66ee52dac067-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-c4ql5\" (UID: \"d6822119-b2f7-4a01-9738-66ee52dac067\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-c4ql5" Jul 07 18:50:36.856301 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:36.856294 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-service-ca-bundle\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:36.856492 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:36.856342 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 18:50:36.856492 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:36.856381 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-metrics-certs podName:2003b9f1-daee-437f-8f44-9ce42ae8f6bd nodeName:}" failed. No retries permitted until 2026-07-07 18:50:44.856371467 +0000 UTC m=+52.275731669 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-metrics-certs") pod "router-default-7dd847dbd5-mq8mz" (UID: "2003b9f1-daee-437f-8f44-9ce42ae8f6bd") : secret "router-metrics-certs-default" not found Jul 07 18:50:36.856492 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:36.856394 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-service-ca-bundle podName:2003b9f1-daee-437f-8f44-9ce42ae8f6bd nodeName:}" failed. No retries permitted until 2026-07-07 18:50:44.856387906 +0000 UTC m=+52.275748108 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-service-ca-bundle") pod "router-default-7dd847dbd5-mq8mz" (UID: "2003b9f1-daee-437f-8f44-9ce42ae8f6bd") : configmap references non-existent config key: service-ca.crt Jul 07 18:50:36.856492 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:36.856395 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 18:50:36.856492 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:36.856447 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6822119-b2f7-4a01-9738-66ee52dac067-samples-operator-tls podName:d6822119-b2f7-4a01-9738-66ee52dac067 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:44.856431939 +0000 UTC m=+52.275792144 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d6822119-b2f7-4a01-9738-66ee52dac067-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-c4ql5" (UID: "d6822119-b2f7-4a01-9738-66ee52dac067") : secret "samples-operator-tls" not found Jul 07 18:50:39.307483 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:39.307424 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-55mgr" event={"ID":"29bdac91-8043-4225-9c92-0d90a7d51003","Type":"ContainerStarted","Data":"50e6da2b128c1983b4a907da041ed41b0475149de15b241a5a5929ff76d07a53"} Jul 07 18:50:39.309497 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:39.309452 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk9lb" event={"ID":"9c701b97-bb01-4f1b-af53-8f310ec406ca","Type":"ContainerStarted","Data":"4a8b32343e21c45986418d0a850ad263087010fa2e022b0ce205f69a3d6f95d9"} Jul 07 18:50:39.311381 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:39.311099 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-blnhp" event={"ID":"1ce43d53-3718-49e6-a60e-ce5ca36877ca","Type":"ContainerStarted","Data":"d28d21a3b3534c35add68f40b175d8adfd5db3f64b1eb46a142fec8a1db2a4f2"} Jul 07 18:50:39.312807 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:39.312498 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" event={"ID":"c3b0a294-e7a3-49fa-a089-92d4bc757e8e","Type":"ContainerStarted","Data":"6b2b667475a55a0974e1c39a2d77a4bf92fab4cc16a7a350e38051a1bd9d1276"} Jul 07 18:50:39.314018 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:39.313989 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mq4f7" event={"ID":"f868dc34-c7da-4e41-abd3-8ad56bae2bc6","Type":"ContainerStarted","Data":"fcb8b020ceb2c5e4e1dfafd9039b3616a3a922c38e25425693f2792c1e6172ad"} Jul 07 18:50:39.315760 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:39.315737 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9s59k" event={"ID":"76ee52c5-2d6b-4c13-b50f-312157fc165f","Type":"ContainerStarted","Data":"b79ec1f7c440a4e0cada1331d0c0e793fcb88de8bf9b63a7b7566b42dcacc235"} Jul 07 18:50:39.317556 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:39.317530 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/0.log" Jul 07 18:50:39.317671 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:39.317565 2564 generic.go:356] "Generic (PLEG): container finished" podID="7a3286ca-3966-48e7-a4d9-9830a7fb1063" containerID="680a316e24ed08eff54c90707eadd39b7aa69ba3448e16def1229228f834a314" exitCode=255 Jul 07 18:50:39.317671 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:39.317607 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-vcwql" event={"ID":"7a3286ca-3966-48e7-a4d9-9830a7fb1063","Type":"ContainerDied","Data":"680a316e24ed08eff54c90707eadd39b7aa69ba3448e16def1229228f834a314"} Jul 07 18:50:39.317779 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:39.317756 2564 scope.go:117] "RemoveContainer" containerID="680a316e24ed08eff54c90707eadd39b7aa69ba3448e16def1229228f834a314" Jul 07 18:50:39.323950 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:39.323912 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-55mgr" podStartSLOduration=9.412773932 podStartE2EDuration="17.323899672s" totalImagesPullingTime="7.91112574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:39.322636382 +0000 UTC m=+46.741996605" watchObservedRunningTime="2026-07-07 18:50:39.323899672 +0000 UTC m=+46.743259875" Jul 07 18:50:39.339374 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:39.339339 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9s59k" podStartSLOduration=2.421645208 podStartE2EDuration="10.339323546s" totalImagesPullingTime="7.917678338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:39.338167196 +0000 UTC m=+46.757527422" watchObservedRunningTime="2026-07-07 18:50:39.339323546 +0000 UTC m=+46.758683768" Jul 07 18:50:39.355015 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:39.354693 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" podStartSLOduration=9.442160121 podStartE2EDuration="17.354677902s" totalImagesPullingTime="7.912517781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:39.354212964 +0000 UTC m=+46.773573192" watchObservedRunningTime="2026-07-07 18:50:39.354677902 +0000 UTC m=+46.774038128" Jul 07 18:50:39.392612 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:39.392491 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk9lb" podStartSLOduration=9.48845849 podStartE2EDuration="17.392472727s" totalImagesPullingTime="7.904014237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:39.39052226 +0000 UTC m=+46.809882486" watchObservedRunningTime="2026-07-07 18:50:39.392472727 +0000 UTC m=+46.811832962" Jul 07 18:50:39.410950 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:39.410907 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-blnhp" podStartSLOduration=9.502456645 podStartE2EDuration="17.410875122s" totalImagesPullingTime="7.908418477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:39.410601936 +0000 UTC m=+46.829962162" watchObservedRunningTime="2026-07-07 18:50:39.410875122 +0000 UTC m=+46.830235347" Jul 07 18:50:39.428662 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:39.428617 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mq4f7" podStartSLOduration=9.528760611 podStartE2EDuration="17.428599552s" totalImagesPullingTime="7.899838941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:39.427073493 +0000 UTC m=+46.846433718" watchObservedRunningTime="2026-07-07 18:50:39.428599552 +0000 UTC m=+46.847959768" Jul 07 18:50:40.323170 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.323144 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/1.log" Jul 07 18:50:40.324131 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.324106 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/0.log" Jul 07 18:50:40.324274 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.324169 2564 generic.go:356] "Generic (PLEG): container finished" podID="7a3286ca-3966-48e7-a4d9-9830a7fb1063" containerID="9142ec5f6175648f149cd178e5653eeac452a82f5f469093a2e7ea0a33b429e5" exitCode=255 Jul 07 18:50:40.324274 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.324262 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-vcwql" event={"ID":"7a3286ca-3966-48e7-a4d9-9830a7fb1063","Type":"ContainerDied","Data":"9142ec5f6175648f149cd178e5653eeac452a82f5f469093a2e7ea0a33b429e5"} Jul 07 18:50:40.324372 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.324345 2564 scope.go:117] "RemoveContainer" containerID="680a316e24ed08eff54c90707eadd39b7aa69ba3448e16def1229228f834a314" Jul 07 18:50:40.325238 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.324899 2564 scope.go:117] "RemoveContainer" containerID="9142ec5f6175648f149cd178e5653eeac452a82f5f469093a2e7ea0a33b429e5" Jul 07 18:50:40.325238 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:40.325064 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-vcwql_openshift-console-operator(7a3286ca-3966-48e7-a4d9-9830a7fb1063)\"" pod="openshift-console-operator/console-operator-68b757fb67-vcwql" podUID="7a3286ca-3966-48e7-a4d9-9830a7fb1063" Jul 07 18:50:40.426160 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.426121 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ps8ws"] Jul 07 18:50:40.461988 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.461963 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ps8ws"] Jul 07 18:50:40.462199 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.462105 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:40.464505 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.464471 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 18:50:40.464647 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.464542 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 18:50:40.464899 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.464877 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lh7s7\"" Jul 07 18:50:40.490512 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.490493 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fb613ac7-eae7-4e21-92ae-66891f614319-crio-socket\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:40.490630 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.490518 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fb613ac7-eae7-4e21-92ae-66891f614319-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:40.490630 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.490542 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kcr2\" (UniqueName: \"kubernetes.io/projected/fb613ac7-eae7-4e21-92ae-66891f614319-kube-api-access-8kcr2\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:40.490630 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.490567 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb613ac7-eae7-4e21-92ae-66891f614319-data-volume\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:40.490630 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.490623 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fb613ac7-eae7-4e21-92ae-66891f614319-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:40.591330 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.591248 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fb613ac7-eae7-4e21-92ae-66891f614319-crio-socket\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:40.591330 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.591285 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fb613ac7-eae7-4e21-92ae-66891f614319-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:40.591330 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.591310 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kcr2\" (UniqueName: \"kubernetes.io/projected/fb613ac7-eae7-4e21-92ae-66891f614319-kube-api-access-8kcr2\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:40.591561 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.591351 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb613ac7-eae7-4e21-92ae-66891f614319-data-volume\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:40.591561 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.591374 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fb613ac7-eae7-4e21-92ae-66891f614319-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:40.591561 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:40.591419 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:50:40.591561 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:40.591478 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb613ac7-eae7-4e21-92ae-66891f614319-insights-runtime-extractor-tls podName:fb613ac7-eae7-4e21-92ae-66891f614319 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:41.091460436 +0000 UTC m=+48.510820658 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fb613ac7-eae7-4e21-92ae-66891f614319-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ps8ws" (UID: "fb613ac7-eae7-4e21-92ae-66891f614319") : secret "insights-runtime-extractor-tls" not found Jul 07 18:50:40.591561 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.591491 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fb613ac7-eae7-4e21-92ae-66891f614319-crio-socket\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:40.591561 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:40.591503 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 18:50:40.591561 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:40.591541 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fb613ac7-eae7-4e21-92ae-66891f614319-kube-rbac-proxy-cm podName:fb613ac7-eae7-4e21-92ae-66891f614319 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:41.09152809 +0000 UTC m=+48.510888292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fb613ac7-eae7-4e21-92ae-66891f614319-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ps8ws" (UID: "fb613ac7-eae7-4e21-92ae-66891f614319") : configmap "kube-rbac-proxy" not found Jul 07 18:50:40.591883 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.591867 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb613ac7-eae7-4e21-92ae-66891f614319-data-volume\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:40.610019 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:40.609986 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kcr2\" (UniqueName: \"kubernetes.io/projected/fb613ac7-eae7-4e21-92ae-66891f614319-kube-api-access-8kcr2\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:41.096315 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:41.096283 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fb613ac7-eae7-4e21-92ae-66891f614319-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:41.096501 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:41.096344 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fb613ac7-eae7-4e21-92ae-66891f614319-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:41.096501 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:41.096430 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:50:41.096501 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:41.096479 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 18:50:41.096501 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:41.096493 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb613ac7-eae7-4e21-92ae-66891f614319-insights-runtime-extractor-tls podName:fb613ac7-eae7-4e21-92ae-66891f614319 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:42.096478646 +0000 UTC m=+49.515838852 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fb613ac7-eae7-4e21-92ae-66891f614319-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ps8ws" (UID: "fb613ac7-eae7-4e21-92ae-66891f614319") : secret "insights-runtime-extractor-tls" not found Jul 07 18:50:41.096695 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:41.096527 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fb613ac7-eae7-4e21-92ae-66891f614319-kube-rbac-proxy-cm podName:fb613ac7-eae7-4e21-92ae-66891f614319 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:42.096515033 +0000 UTC m=+49.515875254 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fb613ac7-eae7-4e21-92ae-66891f614319-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ps8ws" (UID: "fb613ac7-eae7-4e21-92ae-66891f614319") : configmap "kube-rbac-proxy" not found Jul 07 18:50:41.329671 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:41.329645 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/1.log" Jul 07 18:50:41.330106 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:41.330008 2564 scope.go:117] "RemoveContainer" containerID="9142ec5f6175648f149cd178e5653eeac452a82f5f469093a2e7ea0a33b429e5" Jul 07 18:50:41.330213 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:41.330191 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-vcwql_openshift-console-operator(7a3286ca-3966-48e7-a4d9-9830a7fb1063)\"" pod="openshift-console-operator/console-operator-68b757fb67-vcwql" podUID="7a3286ca-3966-48e7-a4d9-9830a7fb1063" Jul 07 18:50:42.105929 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:42.105885 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fb613ac7-eae7-4e21-92ae-66891f614319-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:42.106086 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:42.106006 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 18:50:42.106086 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:42.106077 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fb613ac7-eae7-4e21-92ae-66891f614319-kube-rbac-proxy-cm podName:fb613ac7-eae7-4e21-92ae-66891f614319 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:44.106062633 +0000 UTC m=+51.525422839 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fb613ac7-eae7-4e21-92ae-66891f614319-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ps8ws" (UID: "fb613ac7-eae7-4e21-92ae-66891f614319") : configmap "kube-rbac-proxy" not found Jul 07 18:50:42.106183 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:42.106023 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fb613ac7-eae7-4e21-92ae-66891f614319-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:42.106183 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:42.106108 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:50:42.106183 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:42.106152 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb613ac7-eae7-4e21-92ae-66891f614319-insights-runtime-extractor-tls podName:fb613ac7-eae7-4e21-92ae-66891f614319 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:44.106141429 +0000 UTC m=+51.525501631 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fb613ac7-eae7-4e21-92ae-66891f614319-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ps8ws" (UID: "fb613ac7-eae7-4e21-92ae-66891f614319") : secret "insights-runtime-extractor-tls" not found Jul 07 18:50:42.232159 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:42.232138 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tmlg4_d305832c-a3ec-453e-9135-697b0b80c619/dns-node-resolver/0.log" Jul 07 18:50:43.025163 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:43.025137 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v6tvj_ac705cb6-fdca-4ffe-b46b-7e299b498b3e/node-ca/0.log" Jul 07 18:50:43.638081 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:43.638051 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9s59k_76ee52c5-2d6b-4c13-b50f-312157fc165f/serve-healthcheck-canary/0.log" Jul 07 18:50:44.122704 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:44.122671 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fb613ac7-eae7-4e21-92ae-66891f614319-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:44.123082 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:44.122717 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fb613ac7-eae7-4e21-92ae-66891f614319-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:44.123082 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:44.122815 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:50:44.123082 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:44.122889 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb613ac7-eae7-4e21-92ae-66891f614319-insights-runtime-extractor-tls podName:fb613ac7-eae7-4e21-92ae-66891f614319 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:48.122875284 +0000 UTC m=+55.542235486 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fb613ac7-eae7-4e21-92ae-66891f614319-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ps8ws" (UID: "fb613ac7-eae7-4e21-92ae-66891f614319") : secret "insights-runtime-extractor-tls" not found Jul 07 18:50:44.123305 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:44.123289 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fb613ac7-eae7-4e21-92ae-66891f614319-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:44.827341 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:44.827315 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:50:44.827502 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:44.827460 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 18:50:44.827502 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:44.827479 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c64b8fbfb-nfsfs: secret "image-registry-tls" not found Jul 07 18:50:44.827568 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:44.827527 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls podName:793c0cea-e51b-4fe8-89ba-425492267b6a nodeName:}" failed. No retries permitted until 2026-07-07 18:51:00.827513059 +0000 UTC m=+68.246873262 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls") pod "image-registry-6c64b8fbfb-nfsfs" (UID: "793c0cea-e51b-4fe8-89ba-425492267b6a") : secret "image-registry-tls" not found Jul 07 18:50:44.927648 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:44.927620 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10350357-c651-4b89-8497-7e16a1b6c701-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-b5nnl\" (UID: \"10350357-c651-4b89-8497-7e16a1b6c701\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl" Jul 07 18:50:44.927648 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:44.927652 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23e1f849-e113-4a42-bc58-448388e582ce-metrics-tls\") pod \"dns-default-wvsd8\" (UID: \"23e1f849-e113-4a42-bc58-448388e582ce\") " pod="openshift-dns/dns-default-wvsd8" Jul 07 18:50:44.927903 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:44.927698 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-metrics-certs\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:44.927903 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:44.927721 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d6822119-b2f7-4a01-9738-66ee52dac067-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-c4ql5\" (UID: \"d6822119-b2f7-4a01-9738-66ee52dac067\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-c4ql5" Jul 07 18:50:44.927903 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:44.927740 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-service-ca-bundle\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:50:44.927903 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:44.927749 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:44.927903 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:44.927796 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/10350357-c651-4b89-8497-7e16a1b6c701-cluster-monitoring-operator-tls podName:10350357-c651-4b89-8497-7e16a1b6c701 nodeName:}" failed. No retries permitted until 2026-07-07 18:51:00.927783713 +0000 UTC m=+68.347143915 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/10350357-c651-4b89-8497-7e16a1b6c701-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-b5nnl" (UID: "10350357-c651-4b89-8497-7e16a1b6c701") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:44.927903 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:44.927828 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 18:50:44.927903 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:44.927829 2564 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 18:50:44.927903 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:44.927861 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:50:44.928179 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:44.927875 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6822119-b2f7-4a01-9738-66ee52dac067-samples-operator-tls podName:d6822119-b2f7-4a01-9738-66ee52dac067 nodeName:}" failed. No retries permitted until 2026-07-07 18:51:00.927861456 +0000 UTC m=+68.347221659 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d6822119-b2f7-4a01-9738-66ee52dac067-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-c4ql5" (UID: "d6822119-b2f7-4a01-9738-66ee52dac067") : secret "samples-operator-tls" not found Jul 07 18:50:44.928179 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:44.927940 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-metrics-certs podName:2003b9f1-daee-437f-8f44-9ce42ae8f6bd nodeName:}" failed. No retries permitted until 2026-07-07 18:51:00.927923183 +0000 UTC m=+68.347283389 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-metrics-certs") pod "router-default-7dd847dbd5-mq8mz" (UID: "2003b9f1-daee-437f-8f44-9ce42ae8f6bd") : secret "router-metrics-certs-default" not found Jul 07 18:50:44.928179 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:44.927957 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23e1f849-e113-4a42-bc58-448388e582ce-metrics-tls podName:23e1f849-e113-4a42-bc58-448388e582ce nodeName:}" failed. No retries permitted until 2026-07-07 18:51:00.927947696 +0000 UTC m=+68.347307904 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/23e1f849-e113-4a42-bc58-448388e582ce-metrics-tls") pod "dns-default-wvsd8" (UID: "23e1f849-e113-4a42-bc58-448388e582ce") : secret "dns-default-metrics-tls" not found Jul 07 18:50:44.928179 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:44.927979 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-service-ca-bundle podName:2003b9f1-daee-437f-8f44-9ce42ae8f6bd nodeName:}" failed. No retries permitted until 2026-07-07 18:51:00.927971094 +0000 UTC m=+68.347331302 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-service-ca-bundle") pod "router-default-7dd847dbd5-mq8mz" (UID: "2003b9f1-daee-437f-8f44-9ce42ae8f6bd") : configmap references non-existent config key: service-ca.crt Jul 07 18:50:48.154269 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:48.154237 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fb613ac7-eae7-4e21-92ae-66891f614319-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:48.154676 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:48.154351 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:50:48.154676 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:48.154401 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb613ac7-eae7-4e21-92ae-66891f614319-insights-runtime-extractor-tls podName:fb613ac7-eae7-4e21-92ae-66891f614319 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:56.154387648 +0000 UTC m=+63.573747851 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fb613ac7-eae7-4e21-92ae-66891f614319-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ps8ws" (UID: "fb613ac7-eae7-4e21-92ae-66891f614319") : secret "insights-runtime-extractor-tls" not found Jul 07 18:50:49.304302 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:49.304267 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-vcwql" Jul 07 18:50:49.304886 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:49.304324 2564 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-vcwql" Jul 07 18:50:49.304886 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:49.304725 2564 scope.go:117] "RemoveContainer" containerID="9142ec5f6175648f149cd178e5653eeac452a82f5f469093a2e7ea0a33b429e5" Jul 07 18:50:49.305006 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:50:49.304918 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-vcwql_openshift-console-operator(7a3286ca-3966-48e7-a4d9-9830a7fb1063)\"" pod="openshift-console-operator/console-operator-68b757fb67-vcwql" podUID="7a3286ca-3966-48e7-a4d9-9830a7fb1063" Jul 07 18:50:54.253131 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:54.253106 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2tp24" Jul 07 18:50:56.219464 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:56.219425 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fb613ac7-eae7-4e21-92ae-66891f614319-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:56.221752 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:56.221732 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fb613ac7-eae7-4e21-92ae-66891f614319-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps8ws\" (UID: \"fb613ac7-eae7-4e21-92ae-66891f614319\") " pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:56.375274 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:56.375252 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lh7s7\"" Jul 07 18:50:56.382641 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:56.382618 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ps8ws" Jul 07 18:50:56.502417 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:56.502394 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ps8ws"] Jul 07 18:50:56.505917 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:50:56.505891 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb613ac7_eae7_4e21_92ae_66891f614319.slice/crio-2cb03be9a7f2eba776f000f80155af43f5e00ddeb9ae71f6ad5223abce6ec635 WatchSource:0}: Error finding container 2cb03be9a7f2eba776f000f80155af43f5e00ddeb9ae71f6ad5223abce6ec635: Status 404 returned error can't find the container with id 2cb03be9a7f2eba776f000f80155af43f5e00ddeb9ae71f6ad5223abce6ec635 Jul 07 18:50:57.379033 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:57.378988 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ps8ws" event={"ID":"fb613ac7-eae7-4e21-92ae-66891f614319","Type":"ContainerStarted","Data":"2cb03be9a7f2eba776f000f80155af43f5e00ddeb9ae71f6ad5223abce6ec635"} Jul 07 18:50:58.844453 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:58.844423 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret\") pod \"global-pull-secret-syncer-hslvd\" (UID: \"2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf\") " pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:58.844890 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:58.844475 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs\") pod \"network-metrics-daemon-hjnqt\" (UID: \"bcee6de1-03c6-4175-9e3b-c2f88c977d88\") " pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:50:58.846904 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:58.846872 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 18:50:58.846904 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:58.846890 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 18:50:58.857206 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:58.857185 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bcee6de1-03c6-4175-9e3b-c2f88c977d88-metrics-certs\") pod \"network-metrics-daemon-hjnqt\" (UID: \"bcee6de1-03c6-4175-9e3b-c2f88c977d88\") " pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:50:58.857299 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:58.857209 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf-original-pull-secret\") pod \"global-pull-secret-syncer-hslvd\" (UID: \"2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf\") " pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:58.945808 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:58.945785 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdhvc\" (UniqueName: \"kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc\") pod \"network-check-target-4hhc9\" (UID: \"6f83f082-dd3d-4223-b29b-f3c00bd060cd\") " pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:50:58.948400 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:58.948371 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdhvc\" (UniqueName: \"kubernetes.io/projected/6f83f082-dd3d-4223-b29b-f3c00bd060cd-kube-api-access-sdhvc\") pod \"network-check-target-4hhc9\" (UID: \"6f83f082-dd3d-4223-b29b-f3c00bd060cd\") " pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:50:59.110720 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:59.110652 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l98hl\"" Jul 07 18:50:59.117701 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:59.117677 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-cxlw6\"" Jul 07 18:50:59.118726 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:59.118709 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hjnqt" Jul 07 18:50:59.119730 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:59.119711 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hslvd" Jul 07 18:50:59.124797 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:50:59.124780 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:51:00.104338 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.104277 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4hhc9"] Jul 07 18:51:00.128454 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.128414 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hslvd"] Jul 07 18:51:00.137590 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:51:00.137546 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e3ec6d8_dbb1_4610_b924_9d3fec6db0bf.slice/crio-b5c07b8437f6ad5f309f8f1d9623c37e3d136fde2b674b687e904ad297a1a81a WatchSource:0}: Error finding container b5c07b8437f6ad5f309f8f1d9623c37e3d136fde2b674b687e904ad297a1a81a: Status 404 returned error can't find the container with id b5c07b8437f6ad5f309f8f1d9623c37e3d136fde2b674b687e904ad297a1a81a Jul 07 18:51:00.139741 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.139719 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hjnqt"] Jul 07 18:51:00.142562 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:51:00.142538 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcee6de1_03c6_4175_9e3b_c2f88c977d88.slice/crio-16f9688790a79f495d7af8c0eb9c788fc6ec7130d4747b620223504e4b02d349 WatchSource:0}: Error finding container 16f9688790a79f495d7af8c0eb9c788fc6ec7130d4747b620223504e4b02d349: Status 404 returned error can't find the container with id 16f9688790a79f495d7af8c0eb9c788fc6ec7130d4747b620223504e4b02d349 Jul 07 18:51:00.390247 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.390220 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hslvd" event={"ID":"2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf","Type":"ContainerStarted","Data":"b5c07b8437f6ad5f309f8f1d9623c37e3d136fde2b674b687e904ad297a1a81a"} Jul 07 18:51:00.391298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.391275 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hjnqt" event={"ID":"bcee6de1-03c6-4175-9e3b-c2f88c977d88","Type":"ContainerStarted","Data":"16f9688790a79f495d7af8c0eb9c788fc6ec7130d4747b620223504e4b02d349"} Jul 07 18:51:00.392611 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.392592 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ps8ws" event={"ID":"fb613ac7-eae7-4e21-92ae-66891f614319","Type":"ContainerStarted","Data":"3b4bc97cfe3026f8179733e6b95c6d5a0c9dd19e96cb74c5329af4f236a85929"} Jul 07 18:51:00.394055 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.394021 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4hhc9" event={"ID":"6f83f082-dd3d-4223-b29b-f3c00bd060cd","Type":"ContainerStarted","Data":"15efda63128cca00b8be0e01f3c032997ba2493cadf2dd884f2278160ecf8885"} Jul 07 18:51:00.394146 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.394067 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4hhc9" event={"ID":"6f83f082-dd3d-4223-b29b-f3c00bd060cd","Type":"ContainerStarted","Data":"b007eb84e98e4ea5979fc483a0bc79ccb7c7eb5245323f1c21cf8c0f4586101a"} Jul 07 18:51:00.394204 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.394163 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:51:00.411047 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.411005 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4hhc9" podStartSLOduration=67.410990622 podStartE2EDuration="1m7.410990622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:49:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:00.410130996 +0000 UTC m=+67.829491223" watchObservedRunningTime="2026-07-07 18:51:00.410990622 +0000 UTC m=+67.830350842" Jul 07 18:51:00.860063 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.859958 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:51:00.862747 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.862609 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls\") pod \"image-registry-6c64b8fbfb-nfsfs\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:51:00.961272 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.961197 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-service-ca-bundle\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:51:00.961272 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.961253 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10350357-c651-4b89-8497-7e16a1b6c701-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-b5nnl\" (UID: \"10350357-c651-4b89-8497-7e16a1b6c701\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl" Jul 07 18:51:00.961489 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.961420 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23e1f849-e113-4a42-bc58-448388e582ce-metrics-tls\") pod \"dns-default-wvsd8\" (UID: \"23e1f849-e113-4a42-bc58-448388e582ce\") " pod="openshift-dns/dns-default-wvsd8" Jul 07 18:51:00.961545 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.961510 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-metrics-certs\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:51:00.961621 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.961557 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d6822119-b2f7-4a01-9738-66ee52dac067-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-c4ql5\" (UID: \"d6822119-b2f7-4a01-9738-66ee52dac067\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-c4ql5" Jul 07 18:51:00.961904 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.961862 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-service-ca-bundle\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:51:00.964921 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.964899 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d6822119-b2f7-4a01-9738-66ee52dac067-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-c4ql5\" (UID: \"d6822119-b2f7-4a01-9738-66ee52dac067\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-c4ql5" Jul 07 18:51:00.965362 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.965327 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2003b9f1-daee-437f-8f44-9ce42ae8f6bd-metrics-certs\") pod \"router-default-7dd847dbd5-mq8mz\" (UID: \"2003b9f1-daee-437f-8f44-9ce42ae8f6bd\") " pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:51:00.966657 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.966602 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23e1f849-e113-4a42-bc58-448388e582ce-metrics-tls\") pod \"dns-default-wvsd8\" (UID: \"23e1f849-e113-4a42-bc58-448388e582ce\") " pod="openshift-dns/dns-default-wvsd8" Jul 07 18:51:00.966937 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:00.966896 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/10350357-c651-4b89-8497-7e16a1b6c701-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-b5nnl\" (UID: \"10350357-c651-4b89-8497-7e16a1b6c701\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl" Jul 07 18:51:01.114891 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:01.114857 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wb95j\"" Jul 07 18:51:01.122930 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:01.122909 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:51:01.138503 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:01.138480 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-cnmmb\"" Jul 07 18:51:01.146354 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:01.146338 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl" Jul 07 18:51:01.149603 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:01.149567 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4qmc4\"" Jul 07 18:51:01.158336 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:01.158301 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:51:01.163075 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:01.163043 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-z5dxq\"" Jul 07 18:51:01.163880 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:01.163863 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-c4ql5" Jul 07 18:51:01.192221 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:01.192197 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-b4nbt\"" Jul 07 18:51:01.200247 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:01.200190 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wvsd8" Jul 07 18:51:01.329708 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:01.329501 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c64b8fbfb-nfsfs"] Jul 07 18:51:01.357634 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:01.357560 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl"] Jul 07 18:51:01.382868 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:01.382804 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7dd847dbd5-mq8mz"] Jul 07 18:51:01.401961 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:01.401928 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ps8ws" event={"ID":"fb613ac7-eae7-4e21-92ae-66891f614319","Type":"ContainerStarted","Data":"b79ccbabf8a3a4750e802e35c64e049f2d367a22b75e79468f45d6d23a9d844d"} Jul 07 18:51:01.407070 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:01.407046 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-c4ql5"] Jul 07 18:51:01.427160 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:01.427142 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wvsd8"] Jul 07 18:51:01.488235 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:51:01.488174 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793c0cea_e51b_4fe8_89ba_425492267b6a.slice/crio-966975b593c0f2bf3219b3db663b9a81d53acb18a1be81d6c3396de5c91de8da WatchSource:0}: Error finding container 966975b593c0f2bf3219b3db663b9a81d53acb18a1be81d6c3396de5c91de8da: Status 404 returned error can't find the container with id 966975b593c0f2bf3219b3db663b9a81d53acb18a1be81d6c3396de5c91de8da Jul 07 18:51:01.489395 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:51:01.489250 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10350357_c651_4b89_8497_7e16a1b6c701.slice/crio-1fb2b827e500fd93845e7a6ea9e38eeea1263f82b1e0eae7bcc9a950f9c339bc WatchSource:0}: Error finding container 1fb2b827e500fd93845e7a6ea9e38eeea1263f82b1e0eae7bcc9a950f9c339bc: Status 404 returned error can't find the container with id 1fb2b827e500fd93845e7a6ea9e38eeea1263f82b1e0eae7bcc9a950f9c339bc Jul 07 18:51:01.489820 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:51:01.489782 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2003b9f1_daee_437f_8f44_9ce42ae8f6bd.slice/crio-1b6240dbb04b08515f120fc9a3f5948e1fdbf0286d18dba0db27dc0bca442b52 WatchSource:0}: Error finding container 1b6240dbb04b08515f120fc9a3f5948e1fdbf0286d18dba0db27dc0bca442b52: Status 404 returned error can't find the container with id 1b6240dbb04b08515f120fc9a3f5948e1fdbf0286d18dba0db27dc0bca442b52 Jul 07 18:51:01.492557 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:51:01.492532 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23e1f849_e113_4a42_bc58_448388e582ce.slice/crio-440e69926aa167b836d1c50b133444231fb2979ec69059fefda44eed97a4afb0 WatchSource:0}: Error finding container 440e69926aa167b836d1c50b133444231fb2979ec69059fefda44eed97a4afb0: Status 404 returned error can't find the container with id 440e69926aa167b836d1c50b133444231fb2979ec69059fefda44eed97a4afb0 Jul 07 18:51:02.098086 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:02.098051 2564 scope.go:117] "RemoveContainer" containerID="9142ec5f6175648f149cd178e5653eeac452a82f5f469093a2e7ea0a33b429e5" Jul 07 18:51:02.410415 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:02.410192 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/1.log" Jul 07 18:51:02.410415 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:02.410281 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-vcwql" event={"ID":"7a3286ca-3966-48e7-a4d9-9830a7fb1063","Type":"ContainerStarted","Data":"a0766c80e0248864ac3e18b6574bbe1e50ec65577f49907d655b41a330da411e"} Jul 07 18:51:02.410912 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:02.410784 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-vcwql" Jul 07 18:51:02.413209 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:02.413147 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wvsd8" event={"ID":"23e1f849-e113-4a42-bc58-448388e582ce","Type":"ContainerStarted","Data":"440e69926aa167b836d1c50b133444231fb2979ec69059fefda44eed97a4afb0"} Jul 07 18:51:02.416456 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:02.416432 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-c4ql5" event={"ID":"d6822119-b2f7-4a01-9738-66ee52dac067","Type":"ContainerStarted","Data":"d05c4831100db4abc61e80e291885b69b06006e433827f175f4f93523620924f"} Jul 07 18:51:02.418548 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:02.418525 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl" event={"ID":"10350357-c651-4b89-8497-7e16a1b6c701","Type":"ContainerStarted","Data":"1fb2b827e500fd93845e7a6ea9e38eeea1263f82b1e0eae7bcc9a950f9c339bc"} Jul 07 18:51:02.421634 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:02.421609 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hjnqt" event={"ID":"bcee6de1-03c6-4175-9e3b-c2f88c977d88","Type":"ContainerStarted","Data":"b37514589c0b1e7ce39ac63d25c8d2dda734d31dbf57c95db87a3b2397ec3720"} Jul 07 18:51:02.421722 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:02.421658 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hjnqt" event={"ID":"bcee6de1-03c6-4175-9e3b-c2f88c977d88","Type":"ContainerStarted","Data":"2c7719c054cff7ce70964622c41ae2865c3bcc9954c90b6c6e4b13c3fd4ac9de"} Jul 07 18:51:02.430163 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:02.430140 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" event={"ID":"2003b9f1-daee-437f-8f44-9ce42ae8f6bd","Type":"ContainerStarted","Data":"8b43d58717acde8116dbfe6bc174a4c3cb9cba68c4716f1eadedf5c8bd45bd0d"} Jul 07 18:51:02.430271 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:02.430184 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" event={"ID":"2003b9f1-daee-437f-8f44-9ce42ae8f6bd","Type":"ContainerStarted","Data":"1b6240dbb04b08515f120fc9a3f5948e1fdbf0286d18dba0db27dc0bca442b52"} Jul 07 18:51:02.434338 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:02.434315 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" event={"ID":"793c0cea-e51b-4fe8-89ba-425492267b6a","Type":"ContainerStarted","Data":"57c9bd70a4d13aa9aebba921ce1953052c711708ed02bafd1ef2070d9ba018a4"} Jul 07 18:51:02.434418 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:02.434357 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" event={"ID":"793c0cea-e51b-4fe8-89ba-425492267b6a","Type":"ContainerStarted","Data":"966975b593c0f2bf3219b3db663b9a81d53acb18a1be81d6c3396de5c91de8da"} Jul 07 18:51:02.434837 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:02.434796 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:51:02.447792 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:02.446161 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-vcwql" podStartSLOduration=32.554285881 podStartE2EDuration="40.4461446s" totalImagesPullingTime="7.891858719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:02.429703056 +0000 UTC m=+69.849063281" watchObservedRunningTime="2026-07-07 18:51:02.4461446 +0000 UTC m=+69.865504826" Jul 07 18:51:02.472236 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:02.470926 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hjnqt" podStartSLOduration=68.089220042 podStartE2EDuration="1m9.470908854s" totalImagesPullingTime="1.381688812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:49:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:02.447613933 +0000 UTC m=+69.866974159" watchObservedRunningTime="2026-07-07 18:51:02.470908854 +0000 UTC m=+69.890269080" Jul 07 18:51:02.472236 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:02.471455 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" podStartSLOduration=69.471445976 podStartE2EDuration="1m9.471445976s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:49:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:02.470466918 +0000 UTC m=+69.889827143" watchObservedRunningTime="2026-07-07 18:51:02.471445976 +0000 UTC m=+69.890806200" Jul 07 18:51:02.734602 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:02.734515 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-vcwql" Jul 07 18:51:02.756221 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:02.755286 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" podStartSLOduration=40.755265127 podStartE2EDuration="40.755265127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:02.494624418 +0000 UTC m=+69.913984642" watchObservedRunningTime="2026-07-07 18:51:02.755265127 +0000 UTC m=+70.174625351" Jul 07 18:51:03.159068 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:03.159035 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:51:03.162173 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:03.161987 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:51:03.440211 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:03.440037 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:51:03.441803 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:03.441620 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7dd847dbd5-mq8mz" Jul 07 18:51:07.457901 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.457819 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ps8ws" event={"ID":"fb613ac7-eae7-4e21-92ae-66891f614319","Type":"ContainerStarted","Data":"d411e5841227cf93719813a7e300899156328c772b53d4a28711c5033f9d790e"} Jul 07 18:51:07.459653 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.459626 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wvsd8" event={"ID":"23e1f849-e113-4a42-bc58-448388e582ce","Type":"ContainerStarted","Data":"b3313adec1e0968aedbde573b07070f887b36b6c71f9ce203e7898cab794b212"} Jul 07 18:51:07.459787 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.459677 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wvsd8" Jul 07 18:51:07.459787 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.459699 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wvsd8" event={"ID":"23e1f849-e113-4a42-bc58-448388e582ce","Type":"ContainerStarted","Data":"28646a456076a4b1c636d85fabeeaa6a61f522578a0d25e576fe0accf5b021ad"} Jul 07 18:51:07.461518 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.461493 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-c4ql5" event={"ID":"d6822119-b2f7-4a01-9738-66ee52dac067","Type":"ContainerStarted","Data":"356003d1e45895b4814c726b983353ca580585294f8da3860855498de11aa5c9"} Jul 07 18:51:07.461619 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.461533 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-c4ql5" event={"ID":"d6822119-b2f7-4a01-9738-66ee52dac067","Type":"ContainerStarted","Data":"239457da4c095fabb5075809be5d9d8eb6af0000ad90e41152d5a4dcfdd9a5e8"} Jul 07 18:51:07.463195 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.463168 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl" event={"ID":"10350357-c651-4b89-8497-7e16a1b6c701","Type":"ContainerStarted","Data":"dda8a4e52c1cdfe5dfa7cbe5d8a0878c9f168d99e4f6963ea391e5e325819d62"} Jul 07 18:51:07.464938 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.464909 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hslvd" event={"ID":"2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf","Type":"ContainerStarted","Data":"f1dace908ec90e9b78b8c6c64960483daacf625f8cb33af4b7d0808c4ac14012"} Jul 07 18:51:07.488116 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.488068 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ps8ws" podStartSLOduration=17.171387371 podStartE2EDuration="27.488055706s" totalImagesPullingTime="10.316668335s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:50:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:07.487368369 +0000 UTC m=+74.906728594" watchObservedRunningTime="2026-07-07 18:51:07.488055706 +0000 UTC m=+74.907415961" Jul 07 18:51:07.536283 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.536230 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wvsd8" podStartSLOduration=33.042352084 podStartE2EDuration="38.536217217s" totalImagesPullingTime="5.493865133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:07.535463244 +0000 UTC m=+74.954823469" watchObservedRunningTime="2026-07-07 18:51:07.536217217 +0000 UTC m=+74.955577440" Jul 07 18:51:07.598775 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.598724 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hslvd" podStartSLOduration=67.710387525 podStartE2EDuration="1m14.598703912s" totalImagesPullingTime="6.888316387s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:07.563746911 +0000 UTC m=+74.983107135" watchObservedRunningTime="2026-07-07 18:51:07.598703912 +0000 UTC m=+75.018064139" Jul 07 18:51:07.606646 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.606606 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-b5nnl" podStartSLOduration=40.115411488 podStartE2EDuration="45.606589811s" totalImagesPullingTime="5.491178323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:07.600715244 +0000 UTC m=+75.020075469" watchObservedRunningTime="2026-07-07 18:51:07.606589811 +0000 UTC m=+75.025950032" Jul 07 18:51:07.606839 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.606824 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-qdtn2"] Jul 07 18:51:07.609033 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.609014 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-qdtn2" Jul 07 18:51:07.612174 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.612107 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 18:51:07.612362 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.612342 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 18:51:07.613104 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.613086 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-zhwvv\"" Jul 07 18:51:07.629429 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.629402 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-qdtn2"] Jul 07 18:51:07.638383 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.638336 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-c4ql5" podStartSLOduration=40.193859327 podStartE2EDuration="45.638318793s" totalImagesPullingTime="5.444459466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:07.637395327 +0000 UTC m=+75.056755552" watchObservedRunningTime="2026-07-07 18:51:07.638318793 +0000 UTC m=+75.057679022" Jul 07 18:51:07.722376 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.722308 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b58r\" (UniqueName: \"kubernetes.io/projected/85c2ce25-fee8-43a2-b63c-4ce54bca5057-kube-api-access-8b58r\") pod \"downloads-685bfb5f64-qdtn2\" (UID: \"85c2ce25-fee8-43a2-b63c-4ce54bca5057\") " pod="openshift-console/downloads-685bfb5f64-qdtn2" Jul 07 18:51:07.823486 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.823445 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b58r\" (UniqueName: \"kubernetes.io/projected/85c2ce25-fee8-43a2-b63c-4ce54bca5057-kube-api-access-8b58r\") pod \"downloads-685bfb5f64-qdtn2\" (UID: \"85c2ce25-fee8-43a2-b63c-4ce54bca5057\") " pod="openshift-console/downloads-685bfb5f64-qdtn2" Jul 07 18:51:07.837338 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.837313 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b58r\" (UniqueName: \"kubernetes.io/projected/85c2ce25-fee8-43a2-b63c-4ce54bca5057-kube-api-access-8b58r\") pod \"downloads-685bfb5f64-qdtn2\" (UID: \"85c2ce25-fee8-43a2-b63c-4ce54bca5057\") " pod="openshift-console/downloads-685bfb5f64-qdtn2" Jul 07 18:51:07.840991 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.840969 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9gts2"] Jul 07 18:51:07.846422 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.846405 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9gts2" Jul 07 18:51:07.848750 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.848731 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-mkd7f\"" Jul 07 18:51:07.848839 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.848797 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 07 18:51:07.853723 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.853696 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9gts2"] Jul 07 18:51:07.920545 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.920525 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-qdtn2" Jul 07 18:51:07.924655 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:07.924631 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5c7e6133-de5f-4d57-93dd-d2f45497e09e-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-9gts2\" (UID: \"5c7e6133-de5f-4d57-93dd-d2f45497e09e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9gts2" Jul 07 18:51:08.025864 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:08.025836 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5c7e6133-de5f-4d57-93dd-d2f45497e09e-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-9gts2\" (UID: \"5c7e6133-de5f-4d57-93dd-d2f45497e09e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9gts2" Jul 07 18:51:08.026075 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:51:08.025985 2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 07 18:51:08.026075 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:51:08.026051 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c7e6133-de5f-4d57-93dd-d2f45497e09e-tls-certificates podName:5c7e6133-de5f-4d57-93dd-d2f45497e09e nodeName:}" failed. No retries permitted until 2026-07-07 18:51:08.52603256 +0000 UTC m=+75.945392766 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/5c7e6133-de5f-4d57-93dd-d2f45497e09e-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-9gts2" (UID: "5c7e6133-de5f-4d57-93dd-d2f45497e09e") : secret "prometheus-operator-admission-webhook-tls" not found Jul 07 18:51:08.048101 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:08.048086 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-qdtn2"] Jul 07 18:51:08.050308 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:51:08.050281 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85c2ce25_fee8_43a2_b63c_4ce54bca5057.slice/crio-2219e64f6165a6c3cbe6080ed10f9275705dc1b38e97d65090934eb1ee1f337d WatchSource:0}: Error finding container 2219e64f6165a6c3cbe6080ed10f9275705dc1b38e97d65090934eb1ee1f337d: Status 404 returned error can't find the container with id 2219e64f6165a6c3cbe6080ed10f9275705dc1b38e97d65090934eb1ee1f337d Jul 07 18:51:08.471367 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:08.471324 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-qdtn2" event={"ID":"85c2ce25-fee8-43a2-b63c-4ce54bca5057","Type":"ContainerStarted","Data":"2219e64f6165a6c3cbe6080ed10f9275705dc1b38e97d65090934eb1ee1f337d"} Jul 07 18:51:08.530023 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:08.529993 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5c7e6133-de5f-4d57-93dd-d2f45497e09e-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-9gts2\" (UID: \"5c7e6133-de5f-4d57-93dd-d2f45497e09e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9gts2" Jul 07 18:51:08.532368 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:08.532344 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5c7e6133-de5f-4d57-93dd-d2f45497e09e-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-9gts2\" (UID: \"5c7e6133-de5f-4d57-93dd-d2f45497e09e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9gts2" Jul 07 18:51:08.756737 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:08.756714 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9gts2" Jul 07 18:51:08.911294 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:08.911265 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9gts2"] Jul 07 18:51:08.913691 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:51:08.913653 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c7e6133_de5f_4d57_93dd_d2f45497e09e.slice/crio-cc2062a10c4fe9c4fc7a56c5117c2aad40b8b18e36e57b5f9c99fc225abd03a7 WatchSource:0}: Error finding container cc2062a10c4fe9c4fc7a56c5117c2aad40b8b18e36e57b5f9c99fc225abd03a7: Status 404 returned error can't find the container with id cc2062a10c4fe9c4fc7a56c5117c2aad40b8b18e36e57b5f9c99fc225abd03a7 Jul 07 18:51:09.475458 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:09.475424 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9gts2" event={"ID":"5c7e6133-de5f-4d57-93dd-d2f45497e09e","Type":"ContainerStarted","Data":"cc2062a10c4fe9c4fc7a56c5117c2aad40b8b18e36e57b5f9c99fc225abd03a7"} Jul 07 18:51:11.484671 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:11.484635 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9gts2" event={"ID":"5c7e6133-de5f-4d57-93dd-d2f45497e09e","Type":"ContainerStarted","Data":"cb54d055144ad2804c32d0ea5b577f86a72bc4c8eeda3a5b928c1dfea6f84bbb"} Jul 07 18:51:11.485137 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:11.484689 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9gts2" Jul 07 18:51:11.489079 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:11.489051 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9gts2" Jul 07 18:51:11.500219 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:11.500175 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-9gts2" podStartSLOduration=2.891865974 podStartE2EDuration="4.50016355s" totalImagesPullingTime="1.608297576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:11.499267408 +0000 UTC m=+78.918627633" watchObservedRunningTime="2026-07-07 18:51:11.50016355 +0000 UTC m=+78.919523774" Jul 07 18:51:11.882649 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:11.882619 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq"] Jul 07 18:51:11.896833 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:11.896629 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq"] Jul 07 18:51:11.896833 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:11.896739 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq" Jul 07 18:51:11.899361 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:11.899168 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 07 18:51:11.899361 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:11.899173 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 07 18:51:11.899941 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:11.899923 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-6sh6p\"" Jul 07 18:51:11.900142 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:11.900123 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 18:51:12.056905 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:12.056870 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-4w8fq\" (UID: \"fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq" Jul 07 18:51:12.057065 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:12.056932 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89ll8\" (UniqueName: \"kubernetes.io/projected/fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f-kube-api-access-89ll8\") pod \"prometheus-operator-567f7b8b7c-4w8fq\" (UID: \"fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq" Jul 07 18:51:12.057065 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:12.056963 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-4w8fq\" (UID: \"fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq" Jul 07 18:51:12.057065 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:12.057009 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-4w8fq\" (UID: \"fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq" Jul 07 18:51:12.157797 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:12.157693 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-4w8fq\" (UID: \"fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq" Jul 07 18:51:12.157948 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:12.157807 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-4w8fq\" (UID: \"fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq" Jul 07 18:51:12.157948 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:12.157862 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89ll8\" (UniqueName: \"kubernetes.io/projected/fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f-kube-api-access-89ll8\") pod \"prometheus-operator-567f7b8b7c-4w8fq\" (UID: \"fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq" Jul 07 18:51:12.157948 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:12.157891 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-4w8fq\" (UID: \"fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq" Jul 07 18:51:12.157948 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:51:12.157937 2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 07 18:51:12.158163 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:51:12.158045 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f-prometheus-operator-tls podName:fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f nodeName:}" failed. No retries permitted until 2026-07-07 18:51:12.657986028 +0000 UTC m=+80.077346242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-4w8fq" (UID: "fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f") : secret "prometheus-operator-tls" not found Jul 07 18:51:12.158547 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:12.158523 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-4w8fq\" (UID: \"fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq" Jul 07 18:51:12.160760 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:12.160738 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-4w8fq\" (UID: \"fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq" Jul 07 18:51:12.168662 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:12.168644 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89ll8\" (UniqueName: \"kubernetes.io/projected/fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f-kube-api-access-89ll8\") pod \"prometheus-operator-567f7b8b7c-4w8fq\" (UID: \"fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq" Jul 07 18:51:12.661392 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:12.661351 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-4w8fq\" (UID: \"fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq" Jul 07 18:51:12.664053 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:12.664020 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-4w8fq\" (UID: \"fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq" Jul 07 18:51:12.814462 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:12.814427 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq" Jul 07 18:51:12.964978 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:12.964943 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq"] Jul 07 18:51:12.969634 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:51:12.969547 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb3ae3d6_9b9b_4f4f_900d_81b18e0d854f.slice/crio-aaff0829f922f33c580e3d260cc7320c7eaf6fdb800d12d3d54074e31207116e WatchSource:0}: Error finding container aaff0829f922f33c580e3d260cc7320c7eaf6fdb800d12d3d54074e31207116e: Status 404 returned error can't find the container with id aaff0829f922f33c580e3d260cc7320c7eaf6fdb800d12d3d54074e31207116e Jul 07 18:51:13.495787 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:13.495489 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq" event={"ID":"fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f","Type":"ContainerStarted","Data":"aaff0829f922f33c580e3d260cc7320c7eaf6fdb800d12d3d54074e31207116e"} Jul 07 18:51:15.506930 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:15.506889 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq" event={"ID":"fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f","Type":"ContainerStarted","Data":"1391d0d3673665ef3f82d6ea5e795c65d9708025993abee2b4bea4bab0ba4bb6"} Jul 07 18:51:15.507379 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:15.506953 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq" event={"ID":"fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f","Type":"ContainerStarted","Data":"73db3425e5579e82d738f0b7313157fbc9505fd117e0df18a51e2cc6eda05542"} Jul 07 18:51:15.526064 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:15.524778 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-4w8fq" podStartSLOduration=2.839479125 podStartE2EDuration="4.524762249s" totalImagesPullingTime="1.685283124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:51:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:15.524651571 +0000 UTC m=+82.944011818" watchObservedRunningTime="2026-07-07 18:51:15.524762249 +0000 UTC m=+82.944122470" Jul 07 18:51:17.277927 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.277897 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-r6ghq"] Jul 07 18:51:17.281039 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.281016 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.283102 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.283053 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 18:51:17.283359 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.283340 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 18:51:17.283727 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.283710 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gwjgx\"" Jul 07 18:51:17.283918 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.283890 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 18:51:17.401791 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.401758 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/920236cd-e793-4a32-bce6-ddb1847993bf-metrics-client-ca\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.401941 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.401799 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/920236cd-e793-4a32-bce6-ddb1847993bf-node-exporter-tls\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.401941 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.401826 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/920236cd-e793-4a32-bce6-ddb1847993bf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.401941 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.401896 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/920236cd-e793-4a32-bce6-ddb1847993bf-node-exporter-accelerators-collector-config\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.401941 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.401928 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn8vx\" (UniqueName: \"kubernetes.io/projected/920236cd-e793-4a32-bce6-ddb1847993bf-kube-api-access-sn8vx\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.402146 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.401958 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/920236cd-e793-4a32-bce6-ddb1847993bf-root\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.402146 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.402022 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/920236cd-e793-4a32-bce6-ddb1847993bf-node-exporter-textfile\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.402146 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.402061 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/920236cd-e793-4a32-bce6-ddb1847993bf-sys\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.402146 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.402102 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/920236cd-e793-4a32-bce6-ddb1847993bf-node-exporter-wtmp\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.473718 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.473691 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wvsd8" Jul 07 18:51:17.503191 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.503159 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/920236cd-e793-4a32-bce6-ddb1847993bf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.503344 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.503214 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/920236cd-e793-4a32-bce6-ddb1847993bf-node-exporter-accelerators-collector-config\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.503344 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.503245 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn8vx\" (UniqueName: \"kubernetes.io/projected/920236cd-e793-4a32-bce6-ddb1847993bf-kube-api-access-sn8vx\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.503344 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.503281 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/920236cd-e793-4a32-bce6-ddb1847993bf-root\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.503494 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.503345 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/920236cd-e793-4a32-bce6-ddb1847993bf-root\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.503494 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.503377 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/920236cd-e793-4a32-bce6-ddb1847993bf-node-exporter-textfile\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.503494 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.503405 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/920236cd-e793-4a32-bce6-ddb1847993bf-sys\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.503494 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.503431 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/920236cd-e793-4a32-bce6-ddb1847993bf-node-exporter-wtmp\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.503494 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.503478 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/920236cd-e793-4a32-bce6-ddb1847993bf-metrics-client-ca\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.503689 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.503506 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/920236cd-e793-4a32-bce6-ddb1847993bf-node-exporter-tls\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.503689 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.503561 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/920236cd-e793-4a32-bce6-ddb1847993bf-sys\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.503689 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:51:17.503635 2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 07 18:51:17.503778 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:51:17.503693 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/920236cd-e793-4a32-bce6-ddb1847993bf-node-exporter-tls podName:920236cd-e793-4a32-bce6-ddb1847993bf nodeName:}" failed. No retries permitted until 2026-07-07 18:51:18.003674866 +0000 UTC m=+85.423035069 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/920236cd-e793-4a32-bce6-ddb1847993bf-node-exporter-tls") pod "node-exporter-r6ghq" (UID: "920236cd-e793-4a32-bce6-ddb1847993bf") : secret "node-exporter-tls" not found Jul 07 18:51:17.503833 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.503808 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/920236cd-e793-4a32-bce6-ddb1847993bf-node-exporter-wtmp\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.503880 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.503829 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/920236cd-e793-4a32-bce6-ddb1847993bf-node-exporter-textfile\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.504524 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.504251 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/920236cd-e793-4a32-bce6-ddb1847993bf-node-exporter-accelerators-collector-config\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.504524 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.504487 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/920236cd-e793-4a32-bce6-ddb1847993bf-metrics-client-ca\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.506186 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.506163 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/920236cd-e793-4a32-bce6-ddb1847993bf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:17.515329 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:17.515308 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn8vx\" (UniqueName: \"kubernetes.io/projected/920236cd-e793-4a32-bce6-ddb1847993bf-kube-api-access-sn8vx\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:18.009239 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:18.009203 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/920236cd-e793-4a32-bce6-ddb1847993bf-node-exporter-tls\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:18.012163 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:18.012128 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/920236cd-e793-4a32-bce6-ddb1847993bf-node-exporter-tls\") pod \"node-exporter-r6ghq\" (UID: \"920236cd-e793-4a32-bce6-ddb1847993bf\") " pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:18.192061 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:18.192008 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r6ghq" Jul 07 18:51:18.204714 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:51:18.204689 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod920236cd_e793_4a32_bce6_ddb1847993bf.slice/crio-9dc447c5e5d5675eee6a193a4ee5b20140680198072aafb886553ec0a6e3e425 WatchSource:0}: Error finding container 9dc447c5e5d5675eee6a193a4ee5b20140680198072aafb886553ec0a6e3e425: Status 404 returned error can't find the container with id 9dc447c5e5d5675eee6a193a4ee5b20140680198072aafb886553ec0a6e3e425 Jul 07 18:51:18.523450 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:18.523418 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r6ghq" event={"ID":"920236cd-e793-4a32-bce6-ddb1847993bf","Type":"ContainerStarted","Data":"9dc447c5e5d5675eee6a193a4ee5b20140680198072aafb886553ec0a6e3e425"} Jul 07 18:51:24.449293 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:24.449254 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:51:25.496503 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.496464 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55985f9fdc-9m6gr"] Jul 07 18:51:25.637938 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.637910 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55985f9fdc-9m6gr"] Jul 07 18:51:25.638150 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.638025 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.640633 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.640568 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 18:51:25.640760 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.640747 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 18:51:25.640978 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.640962 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 18:51:25.641381 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.641362 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-flmwv\"" Jul 07 18:51:25.641543 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.641521 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 18:51:25.641942 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.641899 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 18:51:25.651368 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.651348 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 18:51:25.775682 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.775327 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-console-config\") pod \"console-55985f9fdc-9m6gr\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.775682 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.775382 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f67d4c0-57c0-4aa8-84a8-1720187f2855-console-serving-cert\") pod \"console-55985f9fdc-9m6gr\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.775682 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.775412 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f67d4c0-57c0-4aa8-84a8-1720187f2855-console-oauth-config\") pod \"console-55985f9fdc-9m6gr\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.775682 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.775442 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-service-ca\") pod \"console-55985f9fdc-9m6gr\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.775682 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.775470 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-oauth-serving-cert\") pod \"console-55985f9fdc-9m6gr\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.775682 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.775502 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g867\" (UniqueName: \"kubernetes.io/projected/6f67d4c0-57c0-4aa8-84a8-1720187f2855-kube-api-access-5g867\") pod \"console-55985f9fdc-9m6gr\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.775682 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.775569 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-trusted-ca-bundle\") pod \"console-55985f9fdc-9m6gr\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.876148 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.876110 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-console-config\") pod \"console-55985f9fdc-9m6gr\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.876319 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.876157 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f67d4c0-57c0-4aa8-84a8-1720187f2855-console-serving-cert\") pod \"console-55985f9fdc-9m6gr\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.876319 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.876178 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f67d4c0-57c0-4aa8-84a8-1720187f2855-console-oauth-config\") pod \"console-55985f9fdc-9m6gr\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.876319 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.876209 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-service-ca\") pod \"console-55985f9fdc-9m6gr\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.876319 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.876234 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-oauth-serving-cert\") pod \"console-55985f9fdc-9m6gr\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.876319 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.876264 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g867\" (UniqueName: \"kubernetes.io/projected/6f67d4c0-57c0-4aa8-84a8-1720187f2855-kube-api-access-5g867\") pod \"console-55985f9fdc-9m6gr\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.876319 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.876330 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-trusted-ca-bundle\") pod \"console-55985f9fdc-9m6gr\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.877207 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.877141 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-console-config\") pod \"console-55985f9fdc-9m6gr\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.877327 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.877280 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-service-ca\") pod \"console-55985f9fdc-9m6gr\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.877389 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.877366 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-oauth-serving-cert\") pod \"console-55985f9fdc-9m6gr\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.877470 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.877444 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-trusted-ca-bundle\") pod \"console-55985f9fdc-9m6gr\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.879029 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.879008 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f67d4c0-57c0-4aa8-84a8-1720187f2855-console-serving-cert\") pod \"console-55985f9fdc-9m6gr\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.879170 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.879144 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f67d4c0-57c0-4aa8-84a8-1720187f2855-console-oauth-config\") pod \"console-55985f9fdc-9m6gr\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.884738 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.884694 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g867\" (UniqueName: \"kubernetes.io/projected/6f67d4c0-57c0-4aa8-84a8-1720187f2855-kube-api-access-5g867\") pod \"console-55985f9fdc-9m6gr\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:25.951751 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:25.951722 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:26.787928 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:26.787905 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55985f9fdc-9m6gr"] Jul 07 18:51:26.792009 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:51:26.791984 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f67d4c0_57c0_4aa8_84a8_1720187f2855.slice/crio-3a25b107bf11a581d2580739e4fcfbfa7ec0844e732784c07ef3b92441504b29 WatchSource:0}: Error finding container 3a25b107bf11a581d2580739e4fcfbfa7ec0844e732784c07ef3b92441504b29: Status 404 returned error can't find the container with id 3a25b107bf11a581d2580739e4fcfbfa7ec0844e732784c07ef3b92441504b29 Jul 07 18:51:27.565779 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:27.565739 2564 generic.go:356] "Generic (PLEG): container finished" podID="920236cd-e793-4a32-bce6-ddb1847993bf" containerID="dfe6651970d01a58a0b46b5287a40cf87325db41a483f909007ba3a63c69405b" exitCode=0 Jul 07 18:51:27.566520 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:27.566484 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r6ghq" event={"ID":"920236cd-e793-4a32-bce6-ddb1847993bf","Type":"ContainerDied","Data":"dfe6651970d01a58a0b46b5287a40cf87325db41a483f909007ba3a63c69405b"} Jul 07 18:51:27.568749 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:27.568688 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55985f9fdc-9m6gr" event={"ID":"6f67d4c0-57c0-4aa8-84a8-1720187f2855","Type":"ContainerStarted","Data":"3a25b107bf11a581d2580739e4fcfbfa7ec0844e732784c07ef3b92441504b29"} Jul 07 18:51:27.570550 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:27.570524 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-qdtn2" event={"ID":"85c2ce25-fee8-43a2-b63c-4ce54bca5057","Type":"ContainerStarted","Data":"f0e0433de8801ae45710e144464b791d2360d1518fc681d2ee481e938e85aceb"} Jul 07 18:51:27.571287 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:27.571266 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-qdtn2" Jul 07 18:51:27.585197 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:27.584853 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-qdtn2" Jul 07 18:51:27.603032 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:27.602986 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-qdtn2" podStartSLOduration=1.978757158 podStartE2EDuration="20.602967257s" totalImagesPullingTime="18.624210099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:27.602288516 +0000 UTC m=+95.021648741" watchObservedRunningTime="2026-07-07 18:51:27.602967257 +0000 UTC m=+95.022327480" Jul 07 18:51:28.578803 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:28.578548 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r6ghq" event={"ID":"920236cd-e793-4a32-bce6-ddb1847993bf","Type":"ContainerStarted","Data":"c9dceccc42cd8b3e38c2a82ce73a6b31544fb953bbef8a00291426f6a7275a1d"} Jul 07 18:51:28.578803 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:28.578645 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r6ghq" event={"ID":"920236cd-e793-4a32-bce6-ddb1847993bf","Type":"ContainerStarted","Data":"2899f172ff31369ad5b97608fa5be07d9337a0724e96ca26246780c9a5cc4040"} Jul 07 18:51:28.605502 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:28.605444 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-r6ghq" podStartSLOduration=3.037680669 podStartE2EDuration="11.605423383s" totalImagesPullingTime="8.45195223s" totalInitContainerRuntime="115.790484ms" isStatefulPod=true podCreationTimestamp="2026-07-07 18:51:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:28.601730382 +0000 UTC m=+96.021090606" watchObservedRunningTime="2026-07-07 18:51:28.605423383 +0000 UTC m=+96.024783608" Jul 07 18:51:30.556059 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:30.556023 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c64b8fbfb-nfsfs"] Jul 07 18:51:31.404882 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:31.404848 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4hhc9" Jul 07 18:51:31.604929 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:31.604884 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55985f9fdc-9m6gr" event={"ID":"6f67d4c0-57c0-4aa8-84a8-1720187f2855","Type":"ContainerStarted","Data":"cdcfbb67824f41f2589be70d834a6546cbec92dcd079534b27981d1913c3fd50"} Jul 07 18:51:31.624202 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:31.624148 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55985f9fdc-9m6gr" podStartSLOduration=2.712754705 podStartE2EDuration="6.624128133s" totalImagesPullingTime="3.911373428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:51:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:31.622383788 +0000 UTC m=+99.041744019" watchObservedRunningTime="2026-07-07 18:51:31.624128133 +0000 UTC m=+99.043488357" Jul 07 18:51:31.929311 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:31.929274 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55985f9fdc-9m6gr"] Jul 07 18:51:35.952379 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:35.952346 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:45.660585 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:45.660541 2564 generic.go:356] "Generic (PLEG): container finished" podID="9c701b97-bb01-4f1b-af53-8f310ec406ca" containerID="4a8b32343e21c45986418d0a850ad263087010fa2e022b0ce205f69a3d6f95d9" exitCode=0 Jul 07 18:51:45.660985 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:45.660606 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk9lb" event={"ID":"9c701b97-bb01-4f1b-af53-8f310ec406ca","Type":"ContainerDied","Data":"4a8b32343e21c45986418d0a850ad263087010fa2e022b0ce205f69a3d6f95d9"} Jul 07 18:51:45.660985 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:45.660909 2564 scope.go:117] "RemoveContainer" containerID="4a8b32343e21c45986418d0a850ad263087010fa2e022b0ce205f69a3d6f95d9" Jul 07 18:51:46.666255 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:46.666225 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-wk9lb" event={"ID":"9c701b97-bb01-4f1b-af53-8f310ec406ca","Type":"ContainerStarted","Data":"9f77d243f498e6553a4e627fbc49d8c6ec5e006ec9de2021242c430d91f5c7d9"} Jul 07 18:51:50.682890 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:50.682863 2564 generic.go:356] "Generic (PLEG): container finished" podID="1ce43d53-3718-49e6-a60e-ce5ca36877ca" containerID="d28d21a3b3534c35add68f40b175d8adfd5db3f64b1eb46a142fec8a1db2a4f2" exitCode=0 Jul 07 18:51:50.683449 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:50.682917 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-blnhp" event={"ID":"1ce43d53-3718-49e6-a60e-ce5ca36877ca","Type":"ContainerDied","Data":"d28d21a3b3534c35add68f40b175d8adfd5db3f64b1eb46a142fec8a1db2a4f2"} Jul 07 18:51:50.683449 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:50.683216 2564 scope.go:117] "RemoveContainer" containerID="d28d21a3b3534c35add68f40b175d8adfd5db3f64b1eb46a142fec8a1db2a4f2" Jul 07 18:51:51.688869 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:51.688835 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-blnhp" event={"ID":"1ce43d53-3718-49e6-a60e-ce5ca36877ca","Type":"ContainerStarted","Data":"622c886c10a7c346a015e9372b4922341cccc371063d040b70016fc335058a47"} Jul 07 18:51:55.584511 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:55.584471 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" podUID="793c0cea-e51b-4fe8-89ba-425492267b6a" containerName="registry" containerID="cri-o://57c9bd70a4d13aa9aebba921ce1953052c711708ed02bafd1ef2070d9ba018a4" gracePeriod=30 Jul 07 18:51:55.709708 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:55.709680 2564 generic.go:356] "Generic (PLEG): container finished" podID="793c0cea-e51b-4fe8-89ba-425492267b6a" containerID="57c9bd70a4d13aa9aebba921ce1953052c711708ed02bafd1ef2070d9ba018a4" exitCode=0 Jul 07 18:51:55.709798 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:55.709723 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" event={"ID":"793c0cea-e51b-4fe8-89ba-425492267b6a","Type":"ContainerDied","Data":"57c9bd70a4d13aa9aebba921ce1953052c711708ed02bafd1ef2070d9ba018a4"} Jul 07 18:51:55.978562 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:55.978537 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:51:56.005418 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.005390 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfkrx\" (UniqueName: \"kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-kube-api-access-gfkrx\") pod \"793c0cea-e51b-4fe8-89ba-425492267b6a\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " Jul 07 18:51:56.005547 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.005451 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-certificates\") pod \"793c0cea-e51b-4fe8-89ba-425492267b6a\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " Jul 07 18:51:56.005629 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.005603 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/793c0cea-e51b-4fe8-89ba-425492267b6a-installation-pull-secrets\") pod \"793c0cea-e51b-4fe8-89ba-425492267b6a\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " Jul 07 18:51:56.005676 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.005666 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls\") pod \"793c0cea-e51b-4fe8-89ba-425492267b6a\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " Jul 07 18:51:56.005713 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.005693 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-bound-sa-token\") pod \"793c0cea-e51b-4fe8-89ba-425492267b6a\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " Jul 07 18:51:56.005756 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.005720 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/793c0cea-e51b-4fe8-89ba-425492267b6a-ca-trust-extracted\") pod \"793c0cea-e51b-4fe8-89ba-425492267b6a\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " Jul 07 18:51:56.005811 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.005766 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/793c0cea-e51b-4fe8-89ba-425492267b6a-trusted-ca\") pod \"793c0cea-e51b-4fe8-89ba-425492267b6a\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " Jul 07 18:51:56.005811 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.005800 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/793c0cea-e51b-4fe8-89ba-425492267b6a-image-registry-private-configuration\") pod \"793c0cea-e51b-4fe8-89ba-425492267b6a\" (UID: \"793c0cea-e51b-4fe8-89ba-425492267b6a\") " Jul 07 18:51:56.005904 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.005855 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "793c0cea-e51b-4fe8-89ba-425492267b6a" (UID: "793c0cea-e51b-4fe8-89ba-425492267b6a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:51:56.006239 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.006179 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/793c0cea-e51b-4fe8-89ba-425492267b6a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "793c0cea-e51b-4fe8-89ba-425492267b6a" (UID: "793c0cea-e51b-4fe8-89ba-425492267b6a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:51:56.006239 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.006209 2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-certificates\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:51:56.008727 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.008470 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/793c0cea-e51b-4fe8-89ba-425492267b6a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "793c0cea-e51b-4fe8-89ba-425492267b6a" (UID: "793c0cea-e51b-4fe8-89ba-425492267b6a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:51:56.008920 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.008889 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "793c0cea-e51b-4fe8-89ba-425492267b6a" (UID: "793c0cea-e51b-4fe8-89ba-425492267b6a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:51:56.009043 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.008968 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "793c0cea-e51b-4fe8-89ba-425492267b6a" (UID: "793c0cea-e51b-4fe8-89ba-425492267b6a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:51:56.009043 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.009013 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/793c0cea-e51b-4fe8-89ba-425492267b6a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "793c0cea-e51b-4fe8-89ba-425492267b6a" (UID: "793c0cea-e51b-4fe8-89ba-425492267b6a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:51:56.009992 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.009967 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-kube-api-access-gfkrx" (OuterVolumeSpecName: "kube-api-access-gfkrx") pod "793c0cea-e51b-4fe8-89ba-425492267b6a" (UID: "793c0cea-e51b-4fe8-89ba-425492267b6a"). InnerVolumeSpecName "kube-api-access-gfkrx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:51:56.015797 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.015774 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/793c0cea-e51b-4fe8-89ba-425492267b6a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "793c0cea-e51b-4fe8-89ba-425492267b6a" (UID: "793c0cea-e51b-4fe8-89ba-425492267b6a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:51:56.107406 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.107384 2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-registry-tls\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:51:56.107406 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.107407 2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-bound-sa-token\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:51:56.107544 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.107415 2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/793c0cea-e51b-4fe8-89ba-425492267b6a-ca-trust-extracted\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:51:56.107544 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.107423 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/793c0cea-e51b-4fe8-89ba-425492267b6a-trusted-ca\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:51:56.107544 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.107432 2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/793c0cea-e51b-4fe8-89ba-425492267b6a-image-registry-private-configuration\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:51:56.107544 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.107440 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfkrx\" (UniqueName: \"kubernetes.io/projected/793c0cea-e51b-4fe8-89ba-425492267b6a-kube-api-access-gfkrx\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:51:56.107544 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.107450 2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/793c0cea-e51b-4fe8-89ba-425492267b6a-installation-pull-secrets\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:51:56.714400 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.714376 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" Jul 07 18:51:56.714941 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.714370 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c64b8fbfb-nfsfs" event={"ID":"793c0cea-e51b-4fe8-89ba-425492267b6a","Type":"ContainerDied","Data":"966975b593c0f2bf3219b3db663b9a81d53acb18a1be81d6c3396de5c91de8da"} Jul 07 18:51:56.714941 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.714492 2564 scope.go:117] "RemoveContainer" containerID="57c9bd70a4d13aa9aebba921ce1953052c711708ed02bafd1ef2070d9ba018a4" Jul 07 18:51:56.736734 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.736714 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c64b8fbfb-nfsfs"] Jul 07 18:51:56.740954 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:56.740934 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6c64b8fbfb-nfsfs"] Jul 07 18:51:57.102379 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:57.102355 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="793c0cea-e51b-4fe8-89ba-425492267b6a" path="/var/lib/kubelet/pods/793c0cea-e51b-4fe8-89ba-425492267b6a/volumes" Jul 07 18:51:58.637561 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:58.637522 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-55985f9fdc-9m6gr" podUID="6f67d4c0-57c0-4aa8-84a8-1720187f2855" containerName="console" containerID="cri-o://cdcfbb67824f41f2589be70d834a6546cbec92dcd079534b27981d1913c3fd50" gracePeriod=15 Jul 07 18:51:59.038755 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.038733 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:59.129933 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.129908 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-service-ca\") pod \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " Jul 07 18:51:59.130078 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.129980 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f67d4c0-57c0-4aa8-84a8-1720187f2855-console-oauth-config\") pod \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " Jul 07 18:51:59.130078 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.130021 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-console-config\") pod \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " Jul 07 18:51:59.130078 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.130052 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f67d4c0-57c0-4aa8-84a8-1720187f2855-console-serving-cert\") pod \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " Jul 07 18:51:59.130210 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.130081 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-trusted-ca-bundle\") pod \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " Jul 07 18:51:59.130210 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.130109 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5g867\" (UniqueName: \"kubernetes.io/projected/6f67d4c0-57c0-4aa8-84a8-1720187f2855-kube-api-access-5g867\") pod \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " Jul 07 18:51:59.130210 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.130161 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-oauth-serving-cert\") pod \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\" (UID: \"6f67d4c0-57c0-4aa8-84a8-1720187f2855\") " Jul 07 18:51:59.130349 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.130266 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-service-ca" (OuterVolumeSpecName: "service-ca") pod "6f67d4c0-57c0-4aa8-84a8-1720187f2855" (UID: "6f67d4c0-57c0-4aa8-84a8-1720187f2855"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:51:59.130401 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.130385 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-service-ca\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:51:59.130466 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.130441 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-console-config" (OuterVolumeSpecName: "console-config") pod "6f67d4c0-57c0-4aa8-84a8-1720187f2855" (UID: "6f67d4c0-57c0-4aa8-84a8-1720187f2855"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:51:59.130630 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.130531 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6f67d4c0-57c0-4aa8-84a8-1720187f2855" (UID: "6f67d4c0-57c0-4aa8-84a8-1720187f2855"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:51:59.130710 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.130684 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6f67d4c0-57c0-4aa8-84a8-1720187f2855" (UID: "6f67d4c0-57c0-4aa8-84a8-1720187f2855"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:51:59.132472 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.132451 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f67d4c0-57c0-4aa8-84a8-1720187f2855-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6f67d4c0-57c0-4aa8-84a8-1720187f2855" (UID: "6f67d4c0-57c0-4aa8-84a8-1720187f2855"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:51:59.132550 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.132507 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f67d4c0-57c0-4aa8-84a8-1720187f2855-kube-api-access-5g867" (OuterVolumeSpecName: "kube-api-access-5g867") pod "6f67d4c0-57c0-4aa8-84a8-1720187f2855" (UID: "6f67d4c0-57c0-4aa8-84a8-1720187f2855"). InnerVolumeSpecName "kube-api-access-5g867". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:51:59.132608 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.132557 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f67d4c0-57c0-4aa8-84a8-1720187f2855-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6f67d4c0-57c0-4aa8-84a8-1720187f2855" (UID: "6f67d4c0-57c0-4aa8-84a8-1720187f2855"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:51:59.231078 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.231051 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-console-config\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:51:59.231078 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.231075 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6f67d4c0-57c0-4aa8-84a8-1720187f2855-console-serving-cert\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:51:59.231203 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.231086 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-trusted-ca-bundle\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:51:59.231203 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.231095 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5g867\" (UniqueName: \"kubernetes.io/projected/6f67d4c0-57c0-4aa8-84a8-1720187f2855-kube-api-access-5g867\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:51:59.231203 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.231103 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6f67d4c0-57c0-4aa8-84a8-1720187f2855-oauth-serving-cert\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:51:59.231203 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.231111 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6f67d4c0-57c0-4aa8-84a8-1720187f2855-console-oauth-config\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:51:59.727608 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.727570 2564 generic.go:356] "Generic (PLEG): container finished" podID="6f67d4c0-57c0-4aa8-84a8-1720187f2855" containerID="cdcfbb67824f41f2589be70d834a6546cbec92dcd079534b27981d1913c3fd50" exitCode=0 Jul 07 18:51:59.727939 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.727627 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55985f9fdc-9m6gr" Jul 07 18:51:59.727939 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.727650 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55985f9fdc-9m6gr" event={"ID":"6f67d4c0-57c0-4aa8-84a8-1720187f2855","Type":"ContainerDied","Data":"cdcfbb67824f41f2589be70d834a6546cbec92dcd079534b27981d1913c3fd50"} Jul 07 18:51:59.727939 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.727698 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55985f9fdc-9m6gr" event={"ID":"6f67d4c0-57c0-4aa8-84a8-1720187f2855","Type":"ContainerDied","Data":"3a25b107bf11a581d2580739e4fcfbfa7ec0844e732784c07ef3b92441504b29"} Jul 07 18:51:59.727939 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.727728 2564 scope.go:117] "RemoveContainer" containerID="cdcfbb67824f41f2589be70d834a6546cbec92dcd079534b27981d1913c3fd50" Jul 07 18:51:59.736149 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.736133 2564 scope.go:117] "RemoveContainer" containerID="cdcfbb67824f41f2589be70d834a6546cbec92dcd079534b27981d1913c3fd50" Jul 07 18:51:59.736409 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:51:59.736391 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdcfbb67824f41f2589be70d834a6546cbec92dcd079534b27981d1913c3fd50\": container with ID starting with cdcfbb67824f41f2589be70d834a6546cbec92dcd079534b27981d1913c3fd50 not found: ID does not exist" containerID="cdcfbb67824f41f2589be70d834a6546cbec92dcd079534b27981d1913c3fd50" Jul 07 18:51:59.736467 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.736417 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdcfbb67824f41f2589be70d834a6546cbec92dcd079534b27981d1913c3fd50"} err="failed to get container status \"cdcfbb67824f41f2589be70d834a6546cbec92dcd079534b27981d1913c3fd50\": rpc error: code = NotFound desc = could not find container \"cdcfbb67824f41f2589be70d834a6546cbec92dcd079534b27981d1913c3fd50\": container with ID starting with cdcfbb67824f41f2589be70d834a6546cbec92dcd079534b27981d1913c3fd50 not found: ID does not exist" Jul 07 18:51:59.749929 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.748764 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55985f9fdc-9m6gr"] Jul 07 18:51:59.753174 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:51:59.753154 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-55985f9fdc-9m6gr"] Jul 07 18:52:01.102262 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:01.102231 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f67d4c0-57c0-4aa8-84a8-1720187f2855" path="/var/lib/kubelet/pods/6f67d4c0-57c0-4aa8-84a8-1720187f2855/volumes" Jul 07 18:52:10.770863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:10.770819 2564 generic.go:356] "Generic (PLEG): container finished" podID="c3b0a294-e7a3-49fa-a089-92d4bc757e8e" containerID="6b2b667475a55a0974e1c39a2d77a4bf92fab4cc16a7a350e38051a1bd9d1276" exitCode=0 Jul 07 18:52:10.771379 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:10.770899 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" event={"ID":"c3b0a294-e7a3-49fa-a089-92d4bc757e8e","Type":"ContainerDied","Data":"6b2b667475a55a0974e1c39a2d77a4bf92fab4cc16a7a350e38051a1bd9d1276"} Jul 07 18:52:10.771379 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:10.771265 2564 scope.go:117] "RemoveContainer" containerID="6b2b667475a55a0974e1c39a2d77a4bf92fab4cc16a7a350e38051a1bd9d1276" Jul 07 18:52:11.776362 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:11.776328 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-qx6d9" event={"ID":"c3b0a294-e7a3-49fa-a089-92d4bc757e8e","Type":"ContainerStarted","Data":"41d4cc91ccb5002c8c77e040aeac8419adf66ddfd94f4f3bcd3e19b9bfa61c83"} Jul 07 18:52:37.951375 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:37.951344 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67b458d9f9-hz952"] Jul 07 18:52:37.951843 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:37.951669 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="793c0cea-e51b-4fe8-89ba-425492267b6a" containerName="registry" Jul 07 18:52:37.951843 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:37.951687 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="793c0cea-e51b-4fe8-89ba-425492267b6a" containerName="registry" Jul 07 18:52:37.951843 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:37.951699 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f67d4c0-57c0-4aa8-84a8-1720187f2855" containerName="console" Jul 07 18:52:37.951843 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:37.951708 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f67d4c0-57c0-4aa8-84a8-1720187f2855" containerName="console" Jul 07 18:52:37.951843 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:37.951774 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="793c0cea-e51b-4fe8-89ba-425492267b6a" containerName="registry" Jul 07 18:52:37.951843 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:37.951784 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f67d4c0-57c0-4aa8-84a8-1720187f2855" containerName="console" Jul 07 18:52:37.953673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:37.953640 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:37.956346 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:37.956185 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-flmwv\"" Jul 07 18:52:37.956346 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:37.956220 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 18:52:37.957129 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:37.957086 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 18:52:37.957129 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:37.957091 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 18:52:37.957316 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:37.957160 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 18:52:37.957316 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:37.957104 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 18:52:37.961552 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:37.961291 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 18:52:37.965865 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:37.965840 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67b458d9f9-hz952"] Jul 07 18:52:38.101552 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.101526 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/badcd60d-49a4-454d-85ea-0a28672e4379-console-oauth-config\") pod \"console-67b458d9f9-hz952\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.101705 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.101558 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/badcd60d-49a4-454d-85ea-0a28672e4379-console-serving-cert\") pod \"console-67b458d9f9-hz952\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.101705 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.101614 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-oauth-serving-cert\") pod \"console-67b458d9f9-hz952\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.101705 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.101635 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-console-config\") pod \"console-67b458d9f9-hz952\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.101705 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.101657 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-trusted-ca-bundle\") pod \"console-67b458d9f9-hz952\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.101705 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.101673 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctxxs\" (UniqueName: \"kubernetes.io/projected/badcd60d-49a4-454d-85ea-0a28672e4379-kube-api-access-ctxxs\") pod \"console-67b458d9f9-hz952\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.101889 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.101749 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-service-ca\") pod \"console-67b458d9f9-hz952\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.202789 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.202707 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-service-ca\") pod \"console-67b458d9f9-hz952\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.202789 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.202749 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/badcd60d-49a4-454d-85ea-0a28672e4379-console-oauth-config\") pod \"console-67b458d9f9-hz952\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.202789 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.202777 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/badcd60d-49a4-454d-85ea-0a28672e4379-console-serving-cert\") pod \"console-67b458d9f9-hz952\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.203043 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.202821 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-oauth-serving-cert\") pod \"console-67b458d9f9-hz952\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.203043 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.202848 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-console-config\") pod \"console-67b458d9f9-hz952\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.203043 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.202879 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-trusted-ca-bundle\") pod \"console-67b458d9f9-hz952\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.203043 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.202991 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctxxs\" (UniqueName: \"kubernetes.io/projected/badcd60d-49a4-454d-85ea-0a28672e4379-kube-api-access-ctxxs\") pod \"console-67b458d9f9-hz952\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.203480 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.203458 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-service-ca\") pod \"console-67b458d9f9-hz952\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.203783 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.203764 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-trusted-ca-bundle\") pod \"console-67b458d9f9-hz952\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.203844 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.203778 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-oauth-serving-cert\") pod \"console-67b458d9f9-hz952\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.203844 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.203827 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-console-config\") pod \"console-67b458d9f9-hz952\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.205405 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.205383 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/badcd60d-49a4-454d-85ea-0a28672e4379-console-serving-cert\") pod \"console-67b458d9f9-hz952\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.205515 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.205502 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/badcd60d-49a4-454d-85ea-0a28672e4379-console-oauth-config\") pod \"console-67b458d9f9-hz952\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.215054 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.212659 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctxxs\" (UniqueName: \"kubernetes.io/projected/badcd60d-49a4-454d-85ea-0a28672e4379-kube-api-access-ctxxs\") pod \"console-67b458d9f9-hz952\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.264263 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.264243 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:38.605085 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.604937 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67b458d9f9-hz952"] Jul 07 18:52:38.607877 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:52:38.607849 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbadcd60d_49a4_454d_85ea_0a28672e4379.slice/crio-ecd51bba92d554e326a6f438c1aec0f808bc7cbfb8e5c3d08ef9f07f7855c6ce WatchSource:0}: Error finding container ecd51bba92d554e326a6f438c1aec0f808bc7cbfb8e5c3d08ef9f07f7855c6ce: Status 404 returned error can't find the container with id ecd51bba92d554e326a6f438c1aec0f808bc7cbfb8e5c3d08ef9f07f7855c6ce Jul 07 18:52:38.882514 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.882441 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67b458d9f9-hz952" event={"ID":"badcd60d-49a4-454d-85ea-0a28672e4379","Type":"ContainerStarted","Data":"9a56c84438df7a6f9c039a27aabf38a4d1fbef3d9d80aa13c57fd8f1a4fea044"} Jul 07 18:52:38.882514 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.882486 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67b458d9f9-hz952" event={"ID":"badcd60d-49a4-454d-85ea-0a28672e4379","Type":"ContainerStarted","Data":"ecd51bba92d554e326a6f438c1aec0f808bc7cbfb8e5c3d08ef9f07f7855c6ce"} Jul 07 18:52:38.902396 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:38.902352 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67b458d9f9-hz952" podStartSLOduration=1.902338565 podStartE2EDuration="1.902338565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:52:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:52:38.900431796 +0000 UTC m=+166.319792021" watchObservedRunningTime="2026-07-07 18:52:38.902338565 +0000 UTC m=+166.321698789" Jul 07 18:52:48.264685 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:48.264608 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:48.264685 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:48.264671 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:48.269227 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:48.269209 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:52:48.926154 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:52:48.926127 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:53:23.620630 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:23.620600 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf"] Jul 07 18:53:23.623130 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:23.623114 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf" Jul 07 18:53:23.625961 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:23.625942 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vrm78\"" Jul 07 18:53:23.626161 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:23.626149 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 18:53:23.627007 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:23.626948 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 18:53:23.632476 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:23.632455 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf"] Jul 07 18:53:23.718633 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:23.718608 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnwh2\" (UniqueName: \"kubernetes.io/projected/2f366bc4-38e7-4dd8-8e67-763ffc4f2795-kube-api-access-xnwh2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf\" (UID: \"2f366bc4-38e7-4dd8-8e67-763ffc4f2795\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf" Jul 07 18:53:23.718749 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:23.718640 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f366bc4-38e7-4dd8-8e67-763ffc4f2795-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf\" (UID: \"2f366bc4-38e7-4dd8-8e67-763ffc4f2795\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf" Jul 07 18:53:23.718749 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:23.718727 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f366bc4-38e7-4dd8-8e67-763ffc4f2795-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf\" (UID: \"2f366bc4-38e7-4dd8-8e67-763ffc4f2795\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf" Jul 07 18:53:23.819703 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:23.819680 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f366bc4-38e7-4dd8-8e67-763ffc4f2795-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf\" (UID: \"2f366bc4-38e7-4dd8-8e67-763ffc4f2795\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf" Jul 07 18:53:23.819825 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:23.819724 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnwh2\" (UniqueName: \"kubernetes.io/projected/2f366bc4-38e7-4dd8-8e67-763ffc4f2795-kube-api-access-xnwh2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf\" (UID: \"2f366bc4-38e7-4dd8-8e67-763ffc4f2795\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf" Jul 07 18:53:23.819825 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:23.819744 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f366bc4-38e7-4dd8-8e67-763ffc4f2795-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf\" (UID: \"2f366bc4-38e7-4dd8-8e67-763ffc4f2795\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf" Jul 07 18:53:23.820071 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:23.820050 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f366bc4-38e7-4dd8-8e67-763ffc4f2795-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf\" (UID: \"2f366bc4-38e7-4dd8-8e67-763ffc4f2795\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf" Jul 07 18:53:23.820126 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:23.820096 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f366bc4-38e7-4dd8-8e67-763ffc4f2795-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf\" (UID: \"2f366bc4-38e7-4dd8-8e67-763ffc4f2795\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf" Jul 07 18:53:23.829515 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:23.829494 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnwh2\" (UniqueName: \"kubernetes.io/projected/2f366bc4-38e7-4dd8-8e67-763ffc4f2795-kube-api-access-xnwh2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf\" (UID: \"2f366bc4-38e7-4dd8-8e67-763ffc4f2795\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf" Jul 07 18:53:23.933201 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:23.933152 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf" Jul 07 18:53:24.262691 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:24.262671 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf"] Jul 07 18:53:24.264928 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:53:24.264902 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f366bc4_38e7_4dd8_8e67_763ffc4f2795.slice/crio-b2c429c5124c8c11f5764079ff0a85faae02df7290068110fa0aa30439882304 WatchSource:0}: Error finding container b2c429c5124c8c11f5764079ff0a85faae02df7290068110fa0aa30439882304: Status 404 returned error can't find the container with id b2c429c5124c8c11f5764079ff0a85faae02df7290068110fa0aa30439882304 Jul 07 18:53:25.050548 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:25.050514 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf" event={"ID":"2f366bc4-38e7-4dd8-8e67-763ffc4f2795","Type":"ContainerStarted","Data":"b2c429c5124c8c11f5764079ff0a85faae02df7290068110fa0aa30439882304"} Jul 07 18:53:33.083994 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:33.083958 2564 generic.go:356] "Generic (PLEG): container finished" podID="2f366bc4-38e7-4dd8-8e67-763ffc4f2795" containerID="9e179e8ee45c85bbe4e78e67c3ebb99d8a200ae63a953fb580ac1aa32f80dd02" exitCode=0 Jul 07 18:53:33.084382 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:33.084033 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf" event={"ID":"2f366bc4-38e7-4dd8-8e67-763ffc4f2795","Type":"ContainerDied","Data":"9e179e8ee45c85bbe4e78e67c3ebb99d8a200ae63a953fb580ac1aa32f80dd02"} Jul 07 18:53:35.096494 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:35.096420 2564 generic.go:356] "Generic (PLEG): container finished" podID="2f366bc4-38e7-4dd8-8e67-763ffc4f2795" containerID="c51512f4fa5aad629f0948286dc5fa752df7cab993d0bd5c3bae4645af2b2c14" exitCode=0 Jul 07 18:53:35.096494 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:35.096482 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf" event={"ID":"2f366bc4-38e7-4dd8-8e67-763ffc4f2795","Type":"ContainerDied","Data":"c51512f4fa5aad629f0948286dc5fa752df7cab993d0bd5c3bae4645af2b2c14"} Jul 07 18:53:42.127553 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:42.127524 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf" event={"ID":"2f366bc4-38e7-4dd8-8e67-763ffc4f2795","Type":"ContainerStarted","Data":"7bd4471b125e5b7b15dc886c1aac9a82f971d97de9d18a380306af9cdd748fa2"} Jul 07 18:53:42.146771 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:42.146730 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf" podStartSLOduration=2.411019237 podStartE2EDuration="19.146716005s" totalImagesPullingTime="16.735696768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:53:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:53:42.145994214 +0000 UTC m=+229.565354464" watchObservedRunningTime="2026-07-07 18:53:42.146716005 +0000 UTC m=+229.566076263" Jul 07 18:53:43.132911 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:43.132871 2564 generic.go:356] "Generic (PLEG): container finished" podID="2f366bc4-38e7-4dd8-8e67-763ffc4f2795" containerID="7bd4471b125e5b7b15dc886c1aac9a82f971d97de9d18a380306af9cdd748fa2" exitCode=0 Jul 07 18:53:43.133257 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:43.132926 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf" event={"ID":"2f366bc4-38e7-4dd8-8e67-763ffc4f2795","Type":"ContainerDied","Data":"7bd4471b125e5b7b15dc886c1aac9a82f971d97de9d18a380306af9cdd748fa2"} Jul 07 18:53:44.652693 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:44.652669 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf" Jul 07 18:53:44.770814 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:44.770780 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f366bc4-38e7-4dd8-8e67-763ffc4f2795-bundle\") pod \"2f366bc4-38e7-4dd8-8e67-763ffc4f2795\" (UID: \"2f366bc4-38e7-4dd8-8e67-763ffc4f2795\") " Jul 07 18:53:44.770957 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:44.770830 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnwh2\" (UniqueName: \"kubernetes.io/projected/2f366bc4-38e7-4dd8-8e67-763ffc4f2795-kube-api-access-xnwh2\") pod \"2f366bc4-38e7-4dd8-8e67-763ffc4f2795\" (UID: \"2f366bc4-38e7-4dd8-8e67-763ffc4f2795\") " Jul 07 18:53:44.770957 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:44.770864 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f366bc4-38e7-4dd8-8e67-763ffc4f2795-util\") pod \"2f366bc4-38e7-4dd8-8e67-763ffc4f2795\" (UID: \"2f366bc4-38e7-4dd8-8e67-763ffc4f2795\") " Jul 07 18:53:44.771401 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:44.771378 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f366bc4-38e7-4dd8-8e67-763ffc4f2795-bundle" (OuterVolumeSpecName: "bundle") pod "2f366bc4-38e7-4dd8-8e67-763ffc4f2795" (UID: "2f366bc4-38e7-4dd8-8e67-763ffc4f2795"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:53:44.773039 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:44.773011 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f366bc4-38e7-4dd8-8e67-763ffc4f2795-kube-api-access-xnwh2" (OuterVolumeSpecName: "kube-api-access-xnwh2") pod "2f366bc4-38e7-4dd8-8e67-763ffc4f2795" (UID: "2f366bc4-38e7-4dd8-8e67-763ffc4f2795"). InnerVolumeSpecName "kube-api-access-xnwh2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:53:44.775652 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:44.775625 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f366bc4-38e7-4dd8-8e67-763ffc4f2795-util" (OuterVolumeSpecName: "util") pod "2f366bc4-38e7-4dd8-8e67-763ffc4f2795" (UID: "2f366bc4-38e7-4dd8-8e67-763ffc4f2795"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:53:44.871530 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:44.871503 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2f366bc4-38e7-4dd8-8e67-763ffc4f2795-bundle\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:53:44.871530 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:44.871527 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnwh2\" (UniqueName: \"kubernetes.io/projected/2f366bc4-38e7-4dd8-8e67-763ffc4f2795-kube-api-access-xnwh2\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:53:44.871677 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:44.871536 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2f366bc4-38e7-4dd8-8e67-763ffc4f2795-util\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:53:45.142214 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:45.142189 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf" Jul 07 18:53:45.142317 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:45.142208 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t2vgf" event={"ID":"2f366bc4-38e7-4dd8-8e67-763ffc4f2795","Type":"ContainerDied","Data":"b2c429c5124c8c11f5764079ff0a85faae02df7290068110fa0aa30439882304"} Jul 07 18:53:45.142317 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:45.142260 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b2c429c5124c8c11f5764079ff0a85faae02df7290068110fa0aa30439882304" Jul 07 18:53:51.679080 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:51.679048 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lkk8d"] Jul 07 18:53:51.679492 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:51.679296 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f366bc4-38e7-4dd8-8e67-763ffc4f2795" containerName="extract" Jul 07 18:53:51.679492 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:51.679306 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f366bc4-38e7-4dd8-8e67-763ffc4f2795" containerName="extract" Jul 07 18:53:51.679492 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:51.679324 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f366bc4-38e7-4dd8-8e67-763ffc4f2795" containerName="pull" Jul 07 18:53:51.679492 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:51.679329 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f366bc4-38e7-4dd8-8e67-763ffc4f2795" containerName="pull" Jul 07 18:53:51.679492 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:51.679334 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f366bc4-38e7-4dd8-8e67-763ffc4f2795" containerName="util" Jul 07 18:53:51.679492 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:51.679339 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f366bc4-38e7-4dd8-8e67-763ffc4f2795" containerName="util" Jul 07 18:53:51.679492 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:51.679387 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f366bc4-38e7-4dd8-8e67-763ffc4f2795" containerName="extract" Jul 07 18:53:51.683539 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:51.683520 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lkk8d" Jul 07 18:53:51.685917 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:51.685895 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:53:51.686089 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:51.686063 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-q7xd5\"" Jul 07 18:53:51.686089 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:51.686085 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 07 18:53:51.703923 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:51.703901 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lkk8d"] Jul 07 18:53:51.723057 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:51.723033 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee1aa0f4-eb4f-4ab5-9626-32bfb14358c8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lkk8d\" (UID: \"ee1aa0f4-eb4f-4ab5-9626-32bfb14358c8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lkk8d" Jul 07 18:53:51.723148 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:51.723117 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvxwd\" (UniqueName: \"kubernetes.io/projected/ee1aa0f4-eb4f-4ab5-9626-32bfb14358c8-kube-api-access-cvxwd\") pod \"cert-manager-operator-controller-manager-55c444bf86-lkk8d\" (UID: \"ee1aa0f4-eb4f-4ab5-9626-32bfb14358c8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lkk8d" Jul 07 18:53:51.824171 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:51.824141 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvxwd\" (UniqueName: \"kubernetes.io/projected/ee1aa0f4-eb4f-4ab5-9626-32bfb14358c8-kube-api-access-cvxwd\") pod \"cert-manager-operator-controller-manager-55c444bf86-lkk8d\" (UID: \"ee1aa0f4-eb4f-4ab5-9626-32bfb14358c8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lkk8d" Jul 07 18:53:51.824279 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:51.824178 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee1aa0f4-eb4f-4ab5-9626-32bfb14358c8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lkk8d\" (UID: \"ee1aa0f4-eb4f-4ab5-9626-32bfb14358c8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lkk8d" Jul 07 18:53:51.824504 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:51.824487 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee1aa0f4-eb4f-4ab5-9626-32bfb14358c8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lkk8d\" (UID: \"ee1aa0f4-eb4f-4ab5-9626-32bfb14358c8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lkk8d" Jul 07 18:53:51.834007 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:51.833980 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvxwd\" (UniqueName: \"kubernetes.io/projected/ee1aa0f4-eb4f-4ab5-9626-32bfb14358c8-kube-api-access-cvxwd\") pod \"cert-manager-operator-controller-manager-55c444bf86-lkk8d\" (UID: \"ee1aa0f4-eb4f-4ab5-9626-32bfb14358c8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lkk8d" Jul 07 18:53:51.992753 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:51.992700 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lkk8d" Jul 07 18:53:52.336830 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:52.336808 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lkk8d"] Jul 07 18:53:52.338858 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:53:52.338836 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee1aa0f4_eb4f_4ab5_9626_32bfb14358c8.slice/crio-79255b683de838f52373faf15e309fb0a26350583cbbb68674646ade8402bc8a WatchSource:0}: Error finding container 79255b683de838f52373faf15e309fb0a26350583cbbb68674646ade8402bc8a: Status 404 returned error can't find the container with id 79255b683de838f52373faf15e309fb0a26350583cbbb68674646ade8402bc8a Jul 07 18:53:53.171968 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:53.171929 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lkk8d" event={"ID":"ee1aa0f4-eb4f-4ab5-9626-32bfb14358c8","Type":"ContainerStarted","Data":"79255b683de838f52373faf15e309fb0a26350583cbbb68674646ade8402bc8a"} Jul 07 18:53:54.177542 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:54.177460 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lkk8d" event={"ID":"ee1aa0f4-eb4f-4ab5-9626-32bfb14358c8","Type":"ContainerStarted","Data":"f05b56ef54d81304c56a89d3271cabb4c4fd4001ab7495cf8f6348bf19a0a953"} Jul 07 18:53:54.203403 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:54.203344 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lkk8d" podStartSLOduration=1.6528841330000001 podStartE2EDuration="3.203324857s" totalImagesPullingTime="1.550440724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:53:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:53:54.201590309 +0000 UTC m=+241.620950535" watchObservedRunningTime="2026-07-07 18:53:54.203324857 +0000 UTC m=+241.622685083" Jul 07 18:53:55.475875 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:55.475844 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth"] Jul 07 18:53:55.478484 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:55.478462 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth" Jul 07 18:53:55.480727 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:55.480708 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vrm78\"" Jul 07 18:53:55.480990 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:55.480973 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 18:53:55.482630 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:55.482609 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 18:53:55.489423 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:55.489406 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth"] Jul 07 18:53:55.554227 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:55.554199 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db8c36ef-e0c0-4b78-9eec-e813b514ec91-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth\" (UID: \"db8c36ef-e0c0-4b78-9eec-e813b514ec91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth" Jul 07 18:53:55.554870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:55.554844 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdcms\" (UniqueName: \"kubernetes.io/projected/db8c36ef-e0c0-4b78-9eec-e813b514ec91-kube-api-access-fdcms\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth\" (UID: \"db8c36ef-e0c0-4b78-9eec-e813b514ec91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth" Jul 07 18:53:55.554962 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:55.554888 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db8c36ef-e0c0-4b78-9eec-e813b514ec91-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth\" (UID: \"db8c36ef-e0c0-4b78-9eec-e813b514ec91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth" Jul 07 18:53:55.656002 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:55.655982 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db8c36ef-e0c0-4b78-9eec-e813b514ec91-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth\" (UID: \"db8c36ef-e0c0-4b78-9eec-e813b514ec91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth" Jul 07 18:53:55.656088 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:55.656009 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdcms\" (UniqueName: \"kubernetes.io/projected/db8c36ef-e0c0-4b78-9eec-e813b514ec91-kube-api-access-fdcms\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth\" (UID: \"db8c36ef-e0c0-4b78-9eec-e813b514ec91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth" Jul 07 18:53:55.656088 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:55.656031 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db8c36ef-e0c0-4b78-9eec-e813b514ec91-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth\" (UID: \"db8c36ef-e0c0-4b78-9eec-e813b514ec91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth" Jul 07 18:53:55.656349 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:55.656331 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db8c36ef-e0c0-4b78-9eec-e813b514ec91-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth\" (UID: \"db8c36ef-e0c0-4b78-9eec-e813b514ec91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth" Jul 07 18:53:55.656402 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:55.656376 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db8c36ef-e0c0-4b78-9eec-e813b514ec91-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth\" (UID: \"db8c36ef-e0c0-4b78-9eec-e813b514ec91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth" Jul 07 18:53:55.667273 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:55.667249 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdcms\" (UniqueName: \"kubernetes.io/projected/db8c36ef-e0c0-4b78-9eec-e813b514ec91-kube-api-access-fdcms\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth\" (UID: \"db8c36ef-e0c0-4b78-9eec-e813b514ec91\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth" Jul 07 18:53:55.787495 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:55.787471 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth" Jul 07 18:53:56.125708 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:56.125685 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth"] Jul 07 18:53:56.127522 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:53:56.127496 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb8c36ef_e0c0_4b78_9eec_e813b514ec91.slice/crio-f23149714db9b34c1b1121441e87f3a640a36484bc46fca00c9d8468720c8b71 WatchSource:0}: Error finding container f23149714db9b34c1b1121441e87f3a640a36484bc46fca00c9d8468720c8b71: Status 404 returned error can't find the container with id f23149714db9b34c1b1121441e87f3a640a36484bc46fca00c9d8468720c8b71 Jul 07 18:53:56.186143 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:56.186094 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth" event={"ID":"db8c36ef-e0c0-4b78-9eec-e813b514ec91","Type":"ContainerStarted","Data":"d49fb25c67705ed2d173fcb06a1b7141971b19711f1be7a5bb0a16e9b21ad35e"} Jul 07 18:53:56.186231 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:56.186151 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth" event={"ID":"db8c36ef-e0c0-4b78-9eec-e813b514ec91","Type":"ContainerStarted","Data":"f23149714db9b34c1b1121441e87f3a640a36484bc46fca00c9d8468720c8b71"} Jul 07 18:53:57.191063 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:57.191028 2564 generic.go:356] "Generic (PLEG): container finished" podID="db8c36ef-e0c0-4b78-9eec-e813b514ec91" containerID="d49fb25c67705ed2d173fcb06a1b7141971b19711f1be7a5bb0a16e9b21ad35e" exitCode=0 Jul 07 18:53:57.191437 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:57.191110 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth" event={"ID":"db8c36ef-e0c0-4b78-9eec-e813b514ec91","Type":"ContainerDied","Data":"d49fb25c67705ed2d173fcb06a1b7141971b19711f1be7a5bb0a16e9b21ad35e"} Jul 07 18:53:57.321151 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:57.321123 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-664hf"] Jul 07 18:53:57.325569 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:57.325527 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-664hf" Jul 07 18:53:57.328282 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:57.328258 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 18:53:57.328282 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:57.328257 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-v7snz\"" Jul 07 18:53:57.328469 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:57.328296 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 18:53:57.331727 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:57.331703 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-664hf"] Jul 07 18:53:57.369596 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:57.369548 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/773a9d91-32b7-4e31-b3ca-ed9206b8be10-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-664hf\" (UID: \"773a9d91-32b7-4e31-b3ca-ed9206b8be10\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-664hf" Jul 07 18:53:57.369685 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:57.369629 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85h58\" (UniqueName: \"kubernetes.io/projected/773a9d91-32b7-4e31-b3ca-ed9206b8be10-kube-api-access-85h58\") pod \"cert-manager-cainjector-559cdd8c87-664hf\" (UID: \"773a9d91-32b7-4e31-b3ca-ed9206b8be10\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-664hf" Jul 07 18:53:57.470848 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:57.470784 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85h58\" (UniqueName: \"kubernetes.io/projected/773a9d91-32b7-4e31-b3ca-ed9206b8be10-kube-api-access-85h58\") pod \"cert-manager-cainjector-559cdd8c87-664hf\" (UID: \"773a9d91-32b7-4e31-b3ca-ed9206b8be10\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-664hf" Jul 07 18:53:57.470848 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:57.470825 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/773a9d91-32b7-4e31-b3ca-ed9206b8be10-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-664hf\" (UID: \"773a9d91-32b7-4e31-b3ca-ed9206b8be10\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-664hf" Jul 07 18:53:57.478936 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:57.478919 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/773a9d91-32b7-4e31-b3ca-ed9206b8be10-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-664hf\" (UID: \"773a9d91-32b7-4e31-b3ca-ed9206b8be10\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-664hf" Jul 07 18:53:57.479088 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:57.479071 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-85h58\" (UniqueName: \"kubernetes.io/projected/773a9d91-32b7-4e31-b3ca-ed9206b8be10-kube-api-access-85h58\") pod \"cert-manager-cainjector-559cdd8c87-664hf\" (UID: \"773a9d91-32b7-4e31-b3ca-ed9206b8be10\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-664hf" Jul 07 18:53:57.642372 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:57.642336 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-664hf" Jul 07 18:53:57.996870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:57.996780 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-664hf"] Jul 07 18:53:57.999513 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:53:57.999489 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod773a9d91_32b7_4e31_b3ca_ed9206b8be10.slice/crio-95400197555a2e182c9b661c8e9475ed92ad15cd94cb6461fb1eddda90dfefbc WatchSource:0}: Error finding container 95400197555a2e182c9b661c8e9475ed92ad15cd94cb6461fb1eddda90dfefbc: Status 404 returned error can't find the container with id 95400197555a2e182c9b661c8e9475ed92ad15cd94cb6461fb1eddda90dfefbc Jul 07 18:53:58.197162 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:53:58.197118 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-664hf" event={"ID":"773a9d91-32b7-4e31-b3ca-ed9206b8be10","Type":"ContainerStarted","Data":"95400197555a2e182c9b661c8e9475ed92ad15cd94cb6461fb1eddda90dfefbc"} Jul 07 18:54:00.211176 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:00.211129 2564 generic.go:356] "Generic (PLEG): container finished" podID="db8c36ef-e0c0-4b78-9eec-e813b514ec91" containerID="150311ab458f0afa0a96be5e8044c1cb4649a72f503899dfa4267cb1c2a943a1" exitCode=0 Jul 07 18:54:00.211566 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:00.211240 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth" event={"ID":"db8c36ef-e0c0-4b78-9eec-e813b514ec91","Type":"ContainerDied","Data":"150311ab458f0afa0a96be5e8044c1cb4649a72f503899dfa4267cb1c2a943a1"} Jul 07 18:54:01.217400 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:01.217371 2564 generic.go:356] "Generic (PLEG): container finished" podID="db8c36ef-e0c0-4b78-9eec-e813b514ec91" containerID="f0fe453674f3242ec1614aab0503b6ac8af4394c8babd3f3f2e8b5802890a5d5" exitCode=0 Jul 07 18:54:01.217859 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:01.217451 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth" event={"ID":"db8c36ef-e0c0-4b78-9eec-e813b514ec91","Type":"ContainerDied","Data":"f0fe453674f3242ec1614aab0503b6ac8af4394c8babd3f3f2e8b5802890a5d5"} Jul 07 18:54:01.219036 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:01.219006 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-664hf" event={"ID":"773a9d91-32b7-4e31-b3ca-ed9206b8be10","Type":"ContainerStarted","Data":"57d747aadb07e91654d895bd0e6bd66f307f633ab6fa46e1581f67ca37f18c3d"} Jul 07 18:54:01.254498 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:01.254456 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-664hf" podStartSLOduration=1.77953266 podStartE2EDuration="4.254442483s" totalImagesPullingTime="2.474909823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:53:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:54:01.253085579 +0000 UTC m=+248.672445804" watchObservedRunningTime="2026-07-07 18:54:01.254442483 +0000 UTC m=+248.673802707" Jul 07 18:54:02.703441 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:02.703419 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth" Jul 07 18:54:02.815332 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:02.815307 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdcms\" (UniqueName: \"kubernetes.io/projected/db8c36ef-e0c0-4b78-9eec-e813b514ec91-kube-api-access-fdcms\") pod \"db8c36ef-e0c0-4b78-9eec-e813b514ec91\" (UID: \"db8c36ef-e0c0-4b78-9eec-e813b514ec91\") " Jul 07 18:54:02.815477 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:02.815371 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db8c36ef-e0c0-4b78-9eec-e813b514ec91-bundle\") pod \"db8c36ef-e0c0-4b78-9eec-e813b514ec91\" (UID: \"db8c36ef-e0c0-4b78-9eec-e813b514ec91\") " Jul 07 18:54:02.815477 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:02.815397 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db8c36ef-e0c0-4b78-9eec-e813b514ec91-util\") pod \"db8c36ef-e0c0-4b78-9eec-e813b514ec91\" (UID: \"db8c36ef-e0c0-4b78-9eec-e813b514ec91\") " Jul 07 18:54:02.815803 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:02.815782 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db8c36ef-e0c0-4b78-9eec-e813b514ec91-bundle" (OuterVolumeSpecName: "bundle") pod "db8c36ef-e0c0-4b78-9eec-e813b514ec91" (UID: "db8c36ef-e0c0-4b78-9eec-e813b514ec91"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:54:02.817455 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:02.817425 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db8c36ef-e0c0-4b78-9eec-e813b514ec91-kube-api-access-fdcms" (OuterVolumeSpecName: "kube-api-access-fdcms") pod "db8c36ef-e0c0-4b78-9eec-e813b514ec91" (UID: "db8c36ef-e0c0-4b78-9eec-e813b514ec91"). InnerVolumeSpecName "kube-api-access-fdcms". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:54:02.820113 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:02.820088 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db8c36ef-e0c0-4b78-9eec-e813b514ec91-util" (OuterVolumeSpecName: "util") pod "db8c36ef-e0c0-4b78-9eec-e813b514ec91" (UID: "db8c36ef-e0c0-4b78-9eec-e813b514ec91"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:54:02.916824 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:02.916804 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdcms\" (UniqueName: \"kubernetes.io/projected/db8c36ef-e0c0-4b78-9eec-e813b514ec91-kube-api-access-fdcms\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:54:02.916824 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:02.916825 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/db8c36ef-e0c0-4b78-9eec-e813b514ec91-bundle\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:54:02.916918 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:02.916834 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/db8c36ef-e0c0-4b78-9eec-e813b514ec91-util\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:54:03.228470 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:03.228446 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth" Jul 07 18:54:03.228630 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:03.228417 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87flgjth" event={"ID":"db8c36ef-e0c0-4b78-9eec-e813b514ec91","Type":"ContainerDied","Data":"f23149714db9b34c1b1121441e87f3a640a36484bc46fca00c9d8468720c8b71"} Jul 07 18:54:03.228630 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:03.228592 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f23149714db9b34c1b1121441e87f3a640a36484bc46fca00c9d8468720c8b71" Jul 07 18:54:09.121170 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:09.121134 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl77f"] Jul 07 18:54:09.121615 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:09.121425 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db8c36ef-e0c0-4b78-9eec-e813b514ec91" containerName="util" Jul 07 18:54:09.121615 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:09.121436 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="db8c36ef-e0c0-4b78-9eec-e813b514ec91" containerName="util" Jul 07 18:54:09.121615 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:09.121446 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db8c36ef-e0c0-4b78-9eec-e813b514ec91" containerName="pull" Jul 07 18:54:09.121615 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:09.121451 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="db8c36ef-e0c0-4b78-9eec-e813b514ec91" containerName="pull" Jul 07 18:54:09.121615 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:09.121459 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db8c36ef-e0c0-4b78-9eec-e813b514ec91" containerName="extract" Jul 07 18:54:09.121615 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:09.121467 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="db8c36ef-e0c0-4b78-9eec-e813b514ec91" containerName="extract" Jul 07 18:54:09.121615 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:09.121519 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="db8c36ef-e0c0-4b78-9eec-e813b514ec91" containerName="extract" Jul 07 18:54:09.124549 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:09.124532 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl77f" Jul 07 18:54:09.127077 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:09.127057 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 18:54:09.127970 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:09.127950 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:54:09.128066 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:09.127980 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-vtq6r\"" Jul 07 18:54:09.134209 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:09.134179 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl77f"] Jul 07 18:54:09.261469 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:09.261445 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/738a31b4-ff84-4b3d-b282-fc47a5ac647f-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rl77f\" (UID: \"738a31b4-ff84-4b3d-b282-fc47a5ac647f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl77f" Jul 07 18:54:09.261608 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:09.261504 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbwgs\" (UniqueName: \"kubernetes.io/projected/738a31b4-ff84-4b3d-b282-fc47a5ac647f-kube-api-access-tbwgs\") pod \"openshift-lws-operator-fd8ccff4c-rl77f\" (UID: \"738a31b4-ff84-4b3d-b282-fc47a5ac647f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl77f" Jul 07 18:54:09.362210 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:09.362182 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbwgs\" (UniqueName: \"kubernetes.io/projected/738a31b4-ff84-4b3d-b282-fc47a5ac647f-kube-api-access-tbwgs\") pod \"openshift-lws-operator-fd8ccff4c-rl77f\" (UID: \"738a31b4-ff84-4b3d-b282-fc47a5ac647f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl77f" Jul 07 18:54:09.362317 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:09.362299 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/738a31b4-ff84-4b3d-b282-fc47a5ac647f-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rl77f\" (UID: \"738a31b4-ff84-4b3d-b282-fc47a5ac647f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl77f" Jul 07 18:54:09.362687 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:09.362671 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/738a31b4-ff84-4b3d-b282-fc47a5ac647f-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rl77f\" (UID: \"738a31b4-ff84-4b3d-b282-fc47a5ac647f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl77f" Jul 07 18:54:09.371841 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:09.371788 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbwgs\" (UniqueName: \"kubernetes.io/projected/738a31b4-ff84-4b3d-b282-fc47a5ac647f-kube-api-access-tbwgs\") pod \"openshift-lws-operator-fd8ccff4c-rl77f\" (UID: \"738a31b4-ff84-4b3d-b282-fc47a5ac647f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl77f" Jul 07 18:54:09.433916 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:09.433894 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl77f" Jul 07 18:54:09.772877 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:09.772846 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl77f"] Jul 07 18:54:09.775149 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:54:09.775115 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod738a31b4_ff84_4b3d_b282_fc47a5ac647f.slice/crio-108ab09c232b4e03b0b9f5fa7ff50637113981e59089856ac52a68e56a01d291 WatchSource:0}: Error finding container 108ab09c232b4e03b0b9f5fa7ff50637113981e59089856ac52a68e56a01d291: Status 404 returned error can't find the container with id 108ab09c232b4e03b0b9f5fa7ff50637113981e59089856ac52a68e56a01d291 Jul 07 18:54:10.262054 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:10.262023 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl77f" event={"ID":"738a31b4-ff84-4b3d-b282-fc47a5ac647f","Type":"ContainerStarted","Data":"108ab09c232b4e03b0b9f5fa7ff50637113981e59089856ac52a68e56a01d291"} Jul 07 18:54:12.273202 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:12.273165 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl77f" event={"ID":"738a31b4-ff84-4b3d-b282-fc47a5ac647f","Type":"ContainerStarted","Data":"a7c0d4553f7f0bb989b7cb6f46f0175460acdab4ddd34bd5e993d0f71c7942b2"} Jul 07 18:54:12.291156 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:12.291109 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rl77f" podStartSLOduration=1.012862086 podStartE2EDuration="3.291094098s" totalImagesPullingTime="2.278232012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:54:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:54:12.289424554 +0000 UTC m=+259.708784779" watchObservedRunningTime="2026-07-07 18:54:12.291094098 +0000 UTC m=+259.710454322" Jul 07 18:54:15.050615 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:15.050506 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm"] Jul 07 18:54:15.068697 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:15.068672 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm"] Jul 07 18:54:15.068851 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:15.068786 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm" Jul 07 18:54:15.071278 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:15.071254 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 18:54:15.071384 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:15.071302 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 18:54:15.071384 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:15.071357 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vrm78\"" Jul 07 18:54:15.202234 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:15.202166 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60307dce-7a43-4c48-b111-fc17aa94315e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm\" (UID: \"60307dce-7a43-4c48-b111-fc17aa94315e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm" Jul 07 18:54:15.202234 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:15.202209 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60307dce-7a43-4c48-b111-fc17aa94315e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm\" (UID: \"60307dce-7a43-4c48-b111-fc17aa94315e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm" Jul 07 18:54:15.202444 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:15.202284 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck5m9\" (UniqueName: \"kubernetes.io/projected/60307dce-7a43-4c48-b111-fc17aa94315e-kube-api-access-ck5m9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm\" (UID: \"60307dce-7a43-4c48-b111-fc17aa94315e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm" Jul 07 18:54:15.303593 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:15.303511 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60307dce-7a43-4c48-b111-fc17aa94315e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm\" (UID: \"60307dce-7a43-4c48-b111-fc17aa94315e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm" Jul 07 18:54:15.303593 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:15.303544 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60307dce-7a43-4c48-b111-fc17aa94315e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm\" (UID: \"60307dce-7a43-4c48-b111-fc17aa94315e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm" Jul 07 18:54:15.303757 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:15.303610 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck5m9\" (UniqueName: \"kubernetes.io/projected/60307dce-7a43-4c48-b111-fc17aa94315e-kube-api-access-ck5m9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm\" (UID: \"60307dce-7a43-4c48-b111-fc17aa94315e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm" Jul 07 18:54:15.303881 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:15.303855 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60307dce-7a43-4c48-b111-fc17aa94315e-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm\" (UID: \"60307dce-7a43-4c48-b111-fc17aa94315e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm" Jul 07 18:54:15.303920 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:15.303879 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60307dce-7a43-4c48-b111-fc17aa94315e-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm\" (UID: \"60307dce-7a43-4c48-b111-fc17aa94315e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm" Jul 07 18:54:15.311918 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:15.311895 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck5m9\" (UniqueName: \"kubernetes.io/projected/60307dce-7a43-4c48-b111-fc17aa94315e-kube-api-access-ck5m9\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm\" (UID: \"60307dce-7a43-4c48-b111-fc17aa94315e\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm" Jul 07 18:54:15.378608 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:15.378563 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm" Jul 07 18:54:15.731617 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:15.731596 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm"] Jul 07 18:54:15.733643 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:54:15.733613 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60307dce_7a43_4c48_b111_fc17aa94315e.slice/crio-f1fbba4e9ea9f0f1ce1b0706959ebcd3199be1fa8b1b676159d6982649d947cc WatchSource:0}: Error finding container f1fbba4e9ea9f0f1ce1b0706959ebcd3199be1fa8b1b676159d6982649d947cc: Status 404 returned error can't find the container with id f1fbba4e9ea9f0f1ce1b0706959ebcd3199be1fa8b1b676159d6982649d947cc Jul 07 18:54:16.291131 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:16.291100 2564 generic.go:356] "Generic (PLEG): container finished" podID="60307dce-7a43-4c48-b111-fc17aa94315e" containerID="42ceea31886bd09e70b879b9b1ca53e9cd4340283616d141768c1fbaf66cd3f7" exitCode=0 Jul 07 18:54:16.291485 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:16.291169 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm" event={"ID":"60307dce-7a43-4c48-b111-fc17aa94315e","Type":"ContainerDied","Data":"42ceea31886bd09e70b879b9b1ca53e9cd4340283616d141768c1fbaf66cd3f7"} Jul 07 18:54:16.291485 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:16.291203 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm" event={"ID":"60307dce-7a43-4c48-b111-fc17aa94315e","Type":"ContainerStarted","Data":"f1fbba4e9ea9f0f1ce1b0706959ebcd3199be1fa8b1b676159d6982649d947cc"} Jul 07 18:54:17.297559 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:17.297528 2564 generic.go:356] "Generic (PLEG): container finished" podID="60307dce-7a43-4c48-b111-fc17aa94315e" containerID="03392eebd7b5924618bf762f3e67202b44ed8500c7a22a7454260ca7e26c30f2" exitCode=0 Jul 07 18:54:17.297933 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:17.297618 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm" event={"ID":"60307dce-7a43-4c48-b111-fc17aa94315e","Type":"ContainerDied","Data":"03392eebd7b5924618bf762f3e67202b44ed8500c7a22a7454260ca7e26c30f2"} Jul 07 18:54:18.307658 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:18.307629 2564 generic.go:356] "Generic (PLEG): container finished" podID="60307dce-7a43-4c48-b111-fc17aa94315e" containerID="b0780a5a061ab009114cb29e3edc5a7cc7a942385b80cf5957c7e8d92cf0a0b9" exitCode=0 Jul 07 18:54:18.308011 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:18.307722 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm" event={"ID":"60307dce-7a43-4c48-b111-fc17aa94315e","Type":"ContainerDied","Data":"b0780a5a061ab009114cb29e3edc5a7cc7a942385b80cf5957c7e8d92cf0a0b9"} Jul 07 18:54:19.774987 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:19.774963 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm" Jul 07 18:54:19.938534 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:19.938506 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60307dce-7a43-4c48-b111-fc17aa94315e-util\") pod \"60307dce-7a43-4c48-b111-fc17aa94315e\" (UID: \"60307dce-7a43-4c48-b111-fc17aa94315e\") " Jul 07 18:54:19.938534 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:19.938534 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60307dce-7a43-4c48-b111-fc17aa94315e-bundle\") pod \"60307dce-7a43-4c48-b111-fc17aa94315e\" (UID: \"60307dce-7a43-4c48-b111-fc17aa94315e\") " Jul 07 18:54:19.938772 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:19.938561 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ck5m9\" (UniqueName: \"kubernetes.io/projected/60307dce-7a43-4c48-b111-fc17aa94315e-kube-api-access-ck5m9\") pod \"60307dce-7a43-4c48-b111-fc17aa94315e\" (UID: \"60307dce-7a43-4c48-b111-fc17aa94315e\") " Jul 07 18:54:19.939143 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:19.939118 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60307dce-7a43-4c48-b111-fc17aa94315e-bundle" (OuterVolumeSpecName: "bundle") pod "60307dce-7a43-4c48-b111-fc17aa94315e" (UID: "60307dce-7a43-4c48-b111-fc17aa94315e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:54:19.940707 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:19.940687 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60307dce-7a43-4c48-b111-fc17aa94315e-kube-api-access-ck5m9" (OuterVolumeSpecName: "kube-api-access-ck5m9") pod "60307dce-7a43-4c48-b111-fc17aa94315e" (UID: "60307dce-7a43-4c48-b111-fc17aa94315e"). InnerVolumeSpecName "kube-api-access-ck5m9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:54:19.946030 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:19.945991 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60307dce-7a43-4c48-b111-fc17aa94315e-util" (OuterVolumeSpecName: "util") pod "60307dce-7a43-4c48-b111-fc17aa94315e" (UID: "60307dce-7a43-4c48-b111-fc17aa94315e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:54:20.039841 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:20.039789 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/60307dce-7a43-4c48-b111-fc17aa94315e-util\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:54:20.039841 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:20.039813 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/60307dce-7a43-4c48-b111-fc17aa94315e-bundle\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:54:20.039841 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:20.039825 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ck5m9\" (UniqueName: \"kubernetes.io/projected/60307dce-7a43-4c48-b111-fc17aa94315e-kube-api-access-ck5m9\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:54:20.323425 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:20.323344 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm" Jul 07 18:54:20.323425 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:20.323370 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afzh6nm" event={"ID":"60307dce-7a43-4c48-b111-fc17aa94315e","Type":"ContainerDied","Data":"f1fbba4e9ea9f0f1ce1b0706959ebcd3199be1fa8b1b676159d6982649d947cc"} Jul 07 18:54:20.323632 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:20.323444 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1fbba4e9ea9f0f1ce1b0706959ebcd3199be1fa8b1b676159d6982649d947cc" Jul 07 18:54:31.781634 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:31.781603 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv"] Jul 07 18:54:31.782016 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:31.781884 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60307dce-7a43-4c48-b111-fc17aa94315e" containerName="util" Jul 07 18:54:31.782016 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:31.781896 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="60307dce-7a43-4c48-b111-fc17aa94315e" containerName="util" Jul 07 18:54:31.782016 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:31.781912 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60307dce-7a43-4c48-b111-fc17aa94315e" containerName="extract" Jul 07 18:54:31.782016 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:31.781916 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="60307dce-7a43-4c48-b111-fc17aa94315e" containerName="extract" Jul 07 18:54:31.782016 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:31.781923 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60307dce-7a43-4c48-b111-fc17aa94315e" containerName="pull" Jul 07 18:54:31.782016 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:31.781929 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="60307dce-7a43-4c48-b111-fc17aa94315e" containerName="pull" Jul 07 18:54:31.782016 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:31.781973 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="60307dce-7a43-4c48-b111-fc17aa94315e" containerName="extract" Jul 07 18:54:31.784661 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:31.784645 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" Jul 07 18:54:31.787283 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:31.787257 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 18:54:31.787400 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:31.787297 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 18:54:31.787400 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:31.787346 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gw8f5\"" Jul 07 18:54:31.787998 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:31.787977 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 18:54:31.788128 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:31.788080 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 18:54:31.810375 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:31.810354 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv"] Jul 07 18:54:31.920545 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:31.920515 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c262c19a-4160-4238-927d-1f282a06b415-webhook-cert\") pod \"opendatahub-operator-controller-manager-64dd56d745-xhpqv\" (UID: \"c262c19a-4160-4238-927d-1f282a06b415\") " pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" Jul 07 18:54:31.920690 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:31.920552 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c262c19a-4160-4238-927d-1f282a06b415-tmp\") pod \"opendatahub-operator-controller-manager-64dd56d745-xhpqv\" (UID: \"c262c19a-4160-4238-927d-1f282a06b415\") " pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" Jul 07 18:54:31.920690 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:31.920646 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjxkx\" (UniqueName: \"kubernetes.io/projected/c262c19a-4160-4238-927d-1f282a06b415-kube-api-access-fjxkx\") pod \"opendatahub-operator-controller-manager-64dd56d745-xhpqv\" (UID: \"c262c19a-4160-4238-927d-1f282a06b415\") " pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" Jul 07 18:54:31.920764 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:31.920689 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c262c19a-4160-4238-927d-1f282a06b415-manifests\") pod \"opendatahub-operator-controller-manager-64dd56d745-xhpqv\" (UID: \"c262c19a-4160-4238-927d-1f282a06b415\") " pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" Jul 07 18:54:31.920764 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:31.920709 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c262c19a-4160-4238-927d-1f282a06b415-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64dd56d745-xhpqv\" (UID: \"c262c19a-4160-4238-927d-1f282a06b415\") " pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" Jul 07 18:54:32.021542 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:32.021516 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjxkx\" (UniqueName: \"kubernetes.io/projected/c262c19a-4160-4238-927d-1f282a06b415-kube-api-access-fjxkx\") pod \"opendatahub-operator-controller-manager-64dd56d745-xhpqv\" (UID: \"c262c19a-4160-4238-927d-1f282a06b415\") " pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" Jul 07 18:54:32.021665 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:32.021556 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c262c19a-4160-4238-927d-1f282a06b415-manifests\") pod \"opendatahub-operator-controller-manager-64dd56d745-xhpqv\" (UID: \"c262c19a-4160-4238-927d-1f282a06b415\") " pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" Jul 07 18:54:32.021665 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:32.021597 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c262c19a-4160-4238-927d-1f282a06b415-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64dd56d745-xhpqv\" (UID: \"c262c19a-4160-4238-927d-1f282a06b415\") " pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" Jul 07 18:54:32.021665 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:32.021616 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c262c19a-4160-4238-927d-1f282a06b415-webhook-cert\") pod \"opendatahub-operator-controller-manager-64dd56d745-xhpqv\" (UID: \"c262c19a-4160-4238-927d-1f282a06b415\") " pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" Jul 07 18:54:32.021665 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:32.021639 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c262c19a-4160-4238-927d-1f282a06b415-tmp\") pod \"opendatahub-operator-controller-manager-64dd56d745-xhpqv\" (UID: \"c262c19a-4160-4238-927d-1f282a06b415\") " pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" Jul 07 18:54:32.021993 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:32.021965 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c262c19a-4160-4238-927d-1f282a06b415-manifests\") pod \"opendatahub-operator-controller-manager-64dd56d745-xhpqv\" (UID: \"c262c19a-4160-4238-927d-1f282a06b415\") " pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" Jul 07 18:54:32.023925 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:32.023897 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c262c19a-4160-4238-927d-1f282a06b415-tmp\") pod \"opendatahub-operator-controller-manager-64dd56d745-xhpqv\" (UID: \"c262c19a-4160-4238-927d-1f282a06b415\") " pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" Jul 07 18:54:32.024279 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:32.024260 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c262c19a-4160-4238-927d-1f282a06b415-webhook-cert\") pod \"opendatahub-operator-controller-manager-64dd56d745-xhpqv\" (UID: \"c262c19a-4160-4238-927d-1f282a06b415\") " pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" Jul 07 18:54:32.024320 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:32.024260 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c262c19a-4160-4238-927d-1f282a06b415-apiservice-cert\") pod \"opendatahub-operator-controller-manager-64dd56d745-xhpqv\" (UID: \"c262c19a-4160-4238-927d-1f282a06b415\") " pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" Jul 07 18:54:32.035450 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:32.035394 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjxkx\" (UniqueName: \"kubernetes.io/projected/c262c19a-4160-4238-927d-1f282a06b415-kube-api-access-fjxkx\") pod \"opendatahub-operator-controller-manager-64dd56d745-xhpqv\" (UID: \"c262c19a-4160-4238-927d-1f282a06b415\") " pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" Jul 07 18:54:32.094238 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:32.094213 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" Jul 07 18:54:32.441980 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:32.441876 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv"] Jul 07 18:54:32.444135 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:54:32.444110 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc262c19a_4160_4238_927d_1f282a06b415.slice/crio-bbc8dd28e97f0b4aba67896f19970013c4b8e0bf12d83186e0c0c60da83ad1b3 WatchSource:0}: Error finding container bbc8dd28e97f0b4aba67896f19970013c4b8e0bf12d83186e0c0c60da83ad1b3: Status 404 returned error can't find the container with id bbc8dd28e97f0b4aba67896f19970013c4b8e0bf12d83186e0c0c60da83ad1b3 Jul 07 18:54:33.382432 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:33.382362 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" event={"ID":"c262c19a-4160-4238-927d-1f282a06b415","Type":"ContainerStarted","Data":"bbc8dd28e97f0b4aba67896f19970013c4b8e0bf12d83186e0c0c60da83ad1b3"} Jul 07 18:54:35.394082 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:35.394043 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" event={"ID":"c262c19a-4160-4238-927d-1f282a06b415","Type":"ContainerStarted","Data":"e576a8ddacfc71444941e5cc47b1ac782c38a40b788e957217c15f6dbe79e0ca"} Jul 07 18:54:36.400008 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:36.399975 2564 generic.go:356] "Generic (PLEG): container finished" podID="c262c19a-4160-4238-927d-1f282a06b415" containerID="e576a8ddacfc71444941e5cc47b1ac782c38a40b788e957217c15f6dbe79e0ca" exitCode=0 Jul 07 18:54:36.400428 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:36.400023 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" event={"ID":"c262c19a-4160-4238-927d-1f282a06b415","Type":"ContainerDied","Data":"e576a8ddacfc71444941e5cc47b1ac782c38a40b788e957217c15f6dbe79e0ca"} Jul 07 18:54:37.406314 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:37.406245 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" event={"ID":"c262c19a-4160-4238-927d-1f282a06b415","Type":"ContainerStarted","Data":"b00607a833d64e6269709136ed7d7e7229a6a106ff829483af9c4a344437740e"} Jul 07 18:54:37.406711 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:37.406368 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" Jul 07 18:54:37.431587 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:37.431531 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" podStartSLOduration=2.3589294499999998 podStartE2EDuration="6.431516558s" totalImagesPullingTime="3.020477551s" totalInitContainerRuntime="1.052109557s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:54:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:54:37.430382804 +0000 UTC m=+284.849743061" watchObservedRunningTime="2026-07-07 18:54:37.431516558 +0000 UTC m=+284.850876782" Jul 07 18:54:48.412718 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:48.412685 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-64dd56d745-xhpqv" Jul 07 18:54:51.278079 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.278051 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5598bb98cc-kjl55"] Jul 07 18:54:51.283873 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.283854 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5598bb98cc-kjl55" Jul 07 18:54:51.288543 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.288523 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 18:54:51.288686 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.288661 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 18:54:51.298192 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.298171 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5598bb98cc-kjl55"] Jul 07 18:54:51.356832 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.356808 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltvfq\" (UniqueName: \"kubernetes.io/projected/71c43c90-08d3-4214-96ca-2b87e14e1df8-kube-api-access-ltvfq\") pod \"kube-auth-proxy-5598bb98cc-kjl55\" (UID: \"71c43c90-08d3-4214-96ca-2b87e14e1df8\") " pod="openshift-ingress/kube-auth-proxy-5598bb98cc-kjl55" Jul 07 18:54:51.356926 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.356855 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71c43c90-08d3-4214-96ca-2b87e14e1df8-tls-certs\") pod \"kube-auth-proxy-5598bb98cc-kjl55\" (UID: \"71c43c90-08d3-4214-96ca-2b87e14e1df8\") " pod="openshift-ingress/kube-auth-proxy-5598bb98cc-kjl55" Jul 07 18:54:51.356966 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.356934 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71c43c90-08d3-4214-96ca-2b87e14e1df8-tmp\") pod \"kube-auth-proxy-5598bb98cc-kjl55\" (UID: \"71c43c90-08d3-4214-96ca-2b87e14e1df8\") " pod="openshift-ingress/kube-auth-proxy-5598bb98cc-kjl55" Jul 07 18:54:51.457641 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.457610 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71c43c90-08d3-4214-96ca-2b87e14e1df8-tls-certs\") pod \"kube-auth-proxy-5598bb98cc-kjl55\" (UID: \"71c43c90-08d3-4214-96ca-2b87e14e1df8\") " pod="openshift-ingress/kube-auth-proxy-5598bb98cc-kjl55" Jul 07 18:54:51.457745 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.457662 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71c43c90-08d3-4214-96ca-2b87e14e1df8-tmp\") pod \"kube-auth-proxy-5598bb98cc-kjl55\" (UID: \"71c43c90-08d3-4214-96ca-2b87e14e1df8\") " pod="openshift-ingress/kube-auth-proxy-5598bb98cc-kjl55" Jul 07 18:54:51.457745 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.457695 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltvfq\" (UniqueName: \"kubernetes.io/projected/71c43c90-08d3-4214-96ca-2b87e14e1df8-kube-api-access-ltvfq\") pod \"kube-auth-proxy-5598bb98cc-kjl55\" (UID: \"71c43c90-08d3-4214-96ca-2b87e14e1df8\") " pod="openshift-ingress/kube-auth-proxy-5598bb98cc-kjl55" Jul 07 18:54:51.459983 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.459955 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71c43c90-08d3-4214-96ca-2b87e14e1df8-tmp\") pod \"kube-auth-proxy-5598bb98cc-kjl55\" (UID: \"71c43c90-08d3-4214-96ca-2b87e14e1df8\") " pod="openshift-ingress/kube-auth-proxy-5598bb98cc-kjl55" Jul 07 18:54:51.460214 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.460192 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71c43c90-08d3-4214-96ca-2b87e14e1df8-tls-certs\") pod \"kube-auth-proxy-5598bb98cc-kjl55\" (UID: \"71c43c90-08d3-4214-96ca-2b87e14e1df8\") " pod="openshift-ingress/kube-auth-proxy-5598bb98cc-kjl55" Jul 07 18:54:51.496801 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.496780 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltvfq\" (UniqueName: \"kubernetes.io/projected/71c43c90-08d3-4214-96ca-2b87e14e1df8-kube-api-access-ltvfq\") pod \"kube-auth-proxy-5598bb98cc-kjl55\" (UID: \"71c43c90-08d3-4214-96ca-2b87e14e1df8\") " pod="openshift-ingress/kube-auth-proxy-5598bb98cc-kjl55" Jul 07 18:54:51.592941 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.592881 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5598bb98cc-kjl55" Jul 07 18:54:51.619729 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.619703 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ttzl8"] Jul 07 18:54:51.622435 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.622415 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-ttzl8" Jul 07 18:54:51.624933 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.624912 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-glsw9\"" Jul 07 18:54:51.625031 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.624913 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 07 18:54:51.644433 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.643237 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ttzl8"] Jul 07 18:54:51.659429 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.659225 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p599k\" (UniqueName: \"kubernetes.io/projected/9ec43cc0-930a-4620-b06d-6d4d6a2d4e33-kube-api-access-p599k\") pod \"odh-model-controller-7998c67dc4-ttzl8\" (UID: \"9ec43cc0-930a-4620-b06d-6d4d6a2d4e33\") " pod="opendatahub/odh-model-controller-7998c67dc4-ttzl8" Jul 07 18:54:51.659429 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.659337 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ec43cc0-930a-4620-b06d-6d4d6a2d4e33-cert\") pod \"odh-model-controller-7998c67dc4-ttzl8\" (UID: \"9ec43cc0-930a-4620-b06d-6d4d6a2d4e33\") " pod="opendatahub/odh-model-controller-7998c67dc4-ttzl8" Jul 07 18:54:51.760289 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.760255 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p599k\" (UniqueName: \"kubernetes.io/projected/9ec43cc0-930a-4620-b06d-6d4d6a2d4e33-kube-api-access-p599k\") pod \"odh-model-controller-7998c67dc4-ttzl8\" (UID: \"9ec43cc0-930a-4620-b06d-6d4d6a2d4e33\") " pod="opendatahub/odh-model-controller-7998c67dc4-ttzl8" Jul 07 18:54:51.760435 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.760357 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ec43cc0-930a-4620-b06d-6d4d6a2d4e33-cert\") pod \"odh-model-controller-7998c67dc4-ttzl8\" (UID: \"9ec43cc0-930a-4620-b06d-6d4d6a2d4e33\") " pod="opendatahub/odh-model-controller-7998c67dc4-ttzl8" Jul 07 18:54:51.760498 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:54:51.760482 2564 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 07 18:54:51.760570 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:54:51.760560 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ec43cc0-930a-4620-b06d-6d4d6a2d4e33-cert podName:9ec43cc0-930a-4620-b06d-6d4d6a2d4e33 nodeName:}" failed. No retries permitted until 2026-07-07 18:54:52.260539898 +0000 UTC m=+299.679900103 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9ec43cc0-930a-4620-b06d-6d4d6a2d4e33-cert") pod "odh-model-controller-7998c67dc4-ttzl8" (UID: "9ec43cc0-930a-4620-b06d-6d4d6a2d4e33") : secret "odh-model-controller-webhook-cert" not found Jul 07 18:54:51.779858 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.779829 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p599k\" (UniqueName: \"kubernetes.io/projected/9ec43cc0-930a-4620-b06d-6d4d6a2d4e33-kube-api-access-p599k\") pod \"odh-model-controller-7998c67dc4-ttzl8\" (UID: \"9ec43cc0-930a-4620-b06d-6d4d6a2d4e33\") " pod="opendatahub/odh-model-controller-7998c67dc4-ttzl8" Jul 07 18:54:51.948844 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:51.948819 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5598bb98cc-kjl55"] Jul 07 18:54:51.951418 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:54:51.951393 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71c43c90_08d3_4214_96ca_2b87e14e1df8.slice/crio-31ba7caa7766998cdabb8c847e48ae6da4d5683359bc8777040d37bc3128b058 WatchSource:0}: Error finding container 31ba7caa7766998cdabb8c847e48ae6da4d5683359bc8777040d37bc3128b058: Status 404 returned error can't find the container with id 31ba7caa7766998cdabb8c847e48ae6da4d5683359bc8777040d37bc3128b058 Jul 07 18:54:52.264819 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:52.264795 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ec43cc0-930a-4620-b06d-6d4d6a2d4e33-cert\") pod \"odh-model-controller-7998c67dc4-ttzl8\" (UID: \"9ec43cc0-930a-4620-b06d-6d4d6a2d4e33\") " pod="opendatahub/odh-model-controller-7998c67dc4-ttzl8" Jul 07 18:54:52.267057 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:52.267032 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ec43cc0-930a-4620-b06d-6d4d6a2d4e33-cert\") pod \"odh-model-controller-7998c67dc4-ttzl8\" (UID: \"9ec43cc0-930a-4620-b06d-6d4d6a2d4e33\") " pod="opendatahub/odh-model-controller-7998c67dc4-ttzl8" Jul 07 18:54:52.480769 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:52.480727 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5598bb98cc-kjl55" event={"ID":"71c43c90-08d3-4214-96ca-2b87e14e1df8","Type":"ContainerStarted","Data":"31ba7caa7766998cdabb8c847e48ae6da4d5683359bc8777040d37bc3128b058"} Jul 07 18:54:52.532786 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:52.532716 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-ttzl8" Jul 07 18:54:52.809933 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:52.809854 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-kx4hr"] Jul 07 18:54:52.815169 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:52.813932 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-kx4hr" Jul 07 18:54:52.816894 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:52.816871 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 07 18:54:52.831340 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:52.831316 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-kx4hr"] Jul 07 18:54:52.870779 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:52.870750 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01d9c78d-f139-40a5-beb1-544649d9794c-tls-certs\") pod \"model-serving-api-54d89bd5fc-kx4hr\" (UID: \"01d9c78d-f139-40a5-beb1-544649d9794c\") " pod="opendatahub/model-serving-api-54d89bd5fc-kx4hr" Jul 07 18:54:52.870892 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:52.870795 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2jpg\" (UniqueName: \"kubernetes.io/projected/01d9c78d-f139-40a5-beb1-544649d9794c-kube-api-access-q2jpg\") pod \"model-serving-api-54d89bd5fc-kx4hr\" (UID: \"01d9c78d-f139-40a5-beb1-544649d9794c\") " pod="opendatahub/model-serving-api-54d89bd5fc-kx4hr" Jul 07 18:54:52.894087 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:52.894068 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ttzl8"] Jul 07 18:54:52.895841 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:54:52.895821 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ec43cc0_930a_4620_b06d_6d4d6a2d4e33.slice/crio-74bffe8db62f25946cc3b7ca4b48bf2c3998db0f09a9bf09a92cbc587cac6f1c WatchSource:0}: Error finding container 74bffe8db62f25946cc3b7ca4b48bf2c3998db0f09a9bf09a92cbc587cac6f1c: Status 404 returned error can't find the container with id 74bffe8db62f25946cc3b7ca4b48bf2c3998db0f09a9bf09a92cbc587cac6f1c Jul 07 18:54:52.971199 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:52.971174 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01d9c78d-f139-40a5-beb1-544649d9794c-tls-certs\") pod \"model-serving-api-54d89bd5fc-kx4hr\" (UID: \"01d9c78d-f139-40a5-beb1-544649d9794c\") " pod="opendatahub/model-serving-api-54d89bd5fc-kx4hr" Jul 07 18:54:52.971331 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:52.971211 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2jpg\" (UniqueName: \"kubernetes.io/projected/01d9c78d-f139-40a5-beb1-544649d9794c-kube-api-access-q2jpg\") pod \"model-serving-api-54d89bd5fc-kx4hr\" (UID: \"01d9c78d-f139-40a5-beb1-544649d9794c\") " pod="opendatahub/model-serving-api-54d89bd5fc-kx4hr" Jul 07 18:54:52.973808 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:52.973780 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 07 18:54:52.984522 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:52.984499 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01d9c78d-f139-40a5-beb1-544649d9794c-tls-certs\") pod \"model-serving-api-54d89bd5fc-kx4hr\" (UID: \"01d9c78d-f139-40a5-beb1-544649d9794c\") " pod="opendatahub/model-serving-api-54d89bd5fc-kx4hr" Jul 07 18:54:52.991687 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:52.991641 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2jpg\" (UniqueName: \"kubernetes.io/projected/01d9c78d-f139-40a5-beb1-544649d9794c-kube-api-access-q2jpg\") pod \"model-serving-api-54d89bd5fc-kx4hr\" (UID: \"01d9c78d-f139-40a5-beb1-544649d9794c\") " pod="opendatahub/model-serving-api-54d89bd5fc-kx4hr" Jul 07 18:54:53.001456 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.001426 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/1.log" Jul 07 18:54:53.007118 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.007096 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/1.log" Jul 07 18:54:53.021161 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.021140 2564 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 18:54:53.127488 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.127316 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-kx4hr" Jul 07 18:54:53.192711 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.192683 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5"] Jul 07 18:54:53.195167 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.195146 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.198403 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.198278 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 18:54:53.198403 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.198298 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 18:54:53.198403 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.198282 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 18:54:53.198403 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.198384 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 18:54:53.199031 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.198455 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-h6kqg\"" Jul 07 18:54:53.201559 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.201540 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5"] Jul 07 18:54:53.273029 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.272996 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/14317d37-21eb-4675-8e28-7d9f477bd768-cacerts\") pod \"istiod-openshift-gateway-7574459656-zm2n5\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.273202 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.273040 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/14317d37-21eb-4675-8e28-7d9f477bd768-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-zm2n5\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.273202 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.273080 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/14317d37-21eb-4675-8e28-7d9f477bd768-local-certs\") pod \"istiod-openshift-gateway-7574459656-zm2n5\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.273202 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.273166 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/14317d37-21eb-4675-8e28-7d9f477bd768-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-zm2n5\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.273357 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.273219 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14317d37-21eb-4675-8e28-7d9f477bd768-istio-token\") pod \"istiod-openshift-gateway-7574459656-zm2n5\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.273357 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.273248 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pmnr\" (UniqueName: \"kubernetes.io/projected/14317d37-21eb-4675-8e28-7d9f477bd768-kube-api-access-9pmnr\") pod \"istiod-openshift-gateway-7574459656-zm2n5\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.273357 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.273281 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14317d37-21eb-4675-8e28-7d9f477bd768-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-zm2n5\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.375699 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.374324 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/14317d37-21eb-4675-8e28-7d9f477bd768-local-certs\") pod \"istiod-openshift-gateway-7574459656-zm2n5\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.375699 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.374392 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/14317d37-21eb-4675-8e28-7d9f477bd768-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-zm2n5\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.375699 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.374445 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14317d37-21eb-4675-8e28-7d9f477bd768-istio-token\") pod \"istiod-openshift-gateway-7574459656-zm2n5\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.375699 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.374476 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pmnr\" (UniqueName: \"kubernetes.io/projected/14317d37-21eb-4675-8e28-7d9f477bd768-kube-api-access-9pmnr\") pod \"istiod-openshift-gateway-7574459656-zm2n5\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.375699 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.374510 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14317d37-21eb-4675-8e28-7d9f477bd768-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-zm2n5\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.375699 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.374566 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/14317d37-21eb-4675-8e28-7d9f477bd768-cacerts\") pod \"istiod-openshift-gateway-7574459656-zm2n5\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.375699 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.374607 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/14317d37-21eb-4675-8e28-7d9f477bd768-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-zm2n5\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.375699 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.375229 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/14317d37-21eb-4675-8e28-7d9f477bd768-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-zm2n5\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.381973 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.381490 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/14317d37-21eb-4675-8e28-7d9f477bd768-local-certs\") pod \"istiod-openshift-gateway-7574459656-zm2n5\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.381973 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.381899 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14317d37-21eb-4675-8e28-7d9f477bd768-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-zm2n5\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.382788 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.382703 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/14317d37-21eb-4675-8e28-7d9f477bd768-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-zm2n5\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.382918 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.382831 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/14317d37-21eb-4675-8e28-7d9f477bd768-cacerts\") pod \"istiod-openshift-gateway-7574459656-zm2n5\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.390025 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.389947 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pmnr\" (UniqueName: \"kubernetes.io/projected/14317d37-21eb-4675-8e28-7d9f477bd768-kube-api-access-9pmnr\") pod \"istiod-openshift-gateway-7574459656-zm2n5\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.397429 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.397404 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14317d37-21eb-4675-8e28-7d9f477bd768-istio-token\") pod \"istiod-openshift-gateway-7574459656-zm2n5\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.490081 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.490039 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-ttzl8" event={"ID":"9ec43cc0-930a-4620-b06d-6d4d6a2d4e33","Type":"ContainerStarted","Data":"74bffe8db62f25946cc3b7ca4b48bf2c3998db0f09a9bf09a92cbc587cac6f1c"} Jul 07 18:54:53.506981 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.506951 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:53.554443 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.554412 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-kx4hr"] Jul 07 18:54:53.559417 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:54:53.559390 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01d9c78d_f139_40a5_beb1_544649d9794c.slice/crio-de522308f882ad037d90063dd63b9a54f2c1456d2eadb4d20409ad9a04c0a463 WatchSource:0}: Error finding container de522308f882ad037d90063dd63b9a54f2c1456d2eadb4d20409ad9a04c0a463: Status 404 returned error can't find the container with id de522308f882ad037d90063dd63b9a54f2c1456d2eadb4d20409ad9a04c0a463 Jul 07 18:54:53.895159 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:53.895135 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5"] Jul 07 18:54:54.496368 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:54.496319 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-kx4hr" event={"ID":"01d9c78d-f139-40a5-beb1-544649d9794c","Type":"ContainerStarted","Data":"de522308f882ad037d90063dd63b9a54f2c1456d2eadb4d20409ad9a04c0a463"} Jul 07 18:54:54.500943 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:54:54.500917 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14317d37_21eb_4675_8e28_7d9f477bd768.slice/crio-6bbedc706c72ec3d34591a6b06c6750c1db8d43fe6511f0f462621d771b02536 WatchSource:0}: Error finding container 6bbedc706c72ec3d34591a6b06c6750c1db8d43fe6511f0f462621d771b02536: Status 404 returned error can't find the container with id 6bbedc706c72ec3d34591a6b06c6750c1db8d43fe6511f0f462621d771b02536 Jul 07 18:54:54.503036 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:54.503014 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 18:54:55.504616 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:55.504561 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" event={"ID":"14317d37-21eb-4675-8e28-7d9f477bd768","Type":"ContainerStarted","Data":"6bbedc706c72ec3d34591a6b06c6750c1db8d43fe6511f0f462621d771b02536"} Jul 07 18:54:55.859285 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:55.859260 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 18:54:57.447036 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:57.446997 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 18:54:57.447302 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:57.447081 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 07 18:54:57.516289 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:57.516256 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5598bb98cc-kjl55" event={"ID":"71c43c90-08d3-4214-96ca-2b87e14e1df8","Type":"ContainerStarted","Data":"04db5d54e12508ef1e49b9905d4c54d5b85b6d79bfbae661ebfa61e18a12ef0d"} Jul 07 18:54:57.519221 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:57.519199 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-ttzl8" event={"ID":"9ec43cc0-930a-4620-b06d-6d4d6a2d4e33","Type":"ContainerStarted","Data":"9407d38617329ecfc8ee26ea5c4e2eecd601a5cefe3b628400e0e874a6b03f20"} Jul 07 18:54:57.519319 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:57.519301 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-ttzl8" Jul 07 18:54:57.520662 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:57.520640 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-kx4hr" event={"ID":"01d9c78d-f139-40a5-beb1-544649d9794c","Type":"ContainerStarted","Data":"46236506478c6ae324ec6363ea24f30b85f0174800f1a7bd159363661f2faf3a"} Jul 07 18:54:57.520775 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:57.520763 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-kx4hr" Jul 07 18:54:57.535696 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:57.535535 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5598bb98cc-kjl55" podStartSLOduration=2.631981644 podStartE2EDuration="6.53551815s" totalImagesPullingTime="3.903536506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:54:57.535451928 +0000 UTC m=+304.954812153" watchObservedRunningTime="2026-07-07 18:54:57.53551815 +0000 UTC m=+304.954878375" Jul 07 18:54:57.555611 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:57.555518 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-ttzl8" podStartSLOduration=2.081955887 podStartE2EDuration="6.555503123s" totalImagesPullingTime="4.473547236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:54:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:54:57.55464759 +0000 UTC m=+304.974007865" watchObservedRunningTime="2026-07-07 18:54:57.555503123 +0000 UTC m=+304.974863346" Jul 07 18:54:57.577071 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:57.577016 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-kx4hr" podStartSLOduration=1.757720765 podStartE2EDuration="5.577002231s" totalImagesPullingTime="3.819281466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:54:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:54:57.574648219 +0000 UTC m=+304.994008442" watchObservedRunningTime="2026-07-07 18:54:57.577002231 +0000 UTC m=+304.996362449" Jul 07 18:54:58.527173 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:58.527127 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" event={"ID":"14317d37-21eb-4675-8e28-7d9f477bd768","Type":"ContainerStarted","Data":"6c6134ef71bdb942c077f51c1cf7e301cacf16420e59228cb036dfefb551210b"} Jul 07 18:54:58.527667 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:58.527613 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:54:58.529192 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:58.529166 2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-zm2n5 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 18:54:58.529303 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:58.529211 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" podUID="14317d37-21eb-4675-8e28-7d9f477bd768" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 18:54:58.563884 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:58.563845 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" podStartSLOduration=2.620313587 podStartE2EDuration="5.563830064s" totalImagesPullingTime="2.943516477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:54:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:54:58.561347378 +0000 UTC m=+305.980707602" watchObservedRunningTime="2026-07-07 18:54:58.563830064 +0000 UTC m=+305.983190288" Jul 07 18:54:59.532153 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:54:59.532122 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:55:03.680865 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.680831 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lnm7t"] Jul 07 18:55:03.684320 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.684299 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9ldv7"] Jul 07 18:55:03.684438 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.684400 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lnm7t" Jul 07 18:55:03.686863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.686848 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 07 18:55:03.686863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.686859 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-qqtbw\"" Jul 07 18:55:03.687270 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.687255 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ldv7" Jul 07 18:55:03.689251 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.689236 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 07 18:55:03.689777 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.689755 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-s2twn\"" Jul 07 18:55:03.706444 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.706428 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lnm7t"] Jul 07 18:55:03.708647 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.708628 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9ldv7"] Jul 07 18:55:03.769356 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.769332 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3541e00e-0398-42b2-94f3-805115d3f75d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9ldv7\" (UID: \"3541e00e-0398-42b2-94f3-805115d3f75d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ldv7" Jul 07 18:55:03.769469 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.769360 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8bb75d2e-60c8-4969-ba8d-abfa2cb2a0d1-cert\") pod \"kserve-controller-manager-d6f4b65b8-lnm7t\" (UID: \"8bb75d2e-60c8-4969-ba8d-abfa2cb2a0d1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lnm7t" Jul 07 18:55:03.769469 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.769393 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7tkn\" (UniqueName: \"kubernetes.io/projected/8bb75d2e-60c8-4969-ba8d-abfa2cb2a0d1-kube-api-access-p7tkn\") pod \"kserve-controller-manager-d6f4b65b8-lnm7t\" (UID: \"8bb75d2e-60c8-4969-ba8d-abfa2cb2a0d1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lnm7t" Jul 07 18:55:03.769570 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.769492 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgnh6\" (UniqueName: \"kubernetes.io/projected/3541e00e-0398-42b2-94f3-805115d3f75d-kube-api-access-cgnh6\") pod \"llmisvc-controller-manager-777c6bd7d5-9ldv7\" (UID: \"3541e00e-0398-42b2-94f3-805115d3f75d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ldv7" Jul 07 18:55:03.870151 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.870126 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7tkn\" (UniqueName: \"kubernetes.io/projected/8bb75d2e-60c8-4969-ba8d-abfa2cb2a0d1-kube-api-access-p7tkn\") pod \"kserve-controller-manager-d6f4b65b8-lnm7t\" (UID: \"8bb75d2e-60c8-4969-ba8d-abfa2cb2a0d1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lnm7t" Jul 07 18:55:03.870269 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.870173 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgnh6\" (UniqueName: \"kubernetes.io/projected/3541e00e-0398-42b2-94f3-805115d3f75d-kube-api-access-cgnh6\") pod \"llmisvc-controller-manager-777c6bd7d5-9ldv7\" (UID: \"3541e00e-0398-42b2-94f3-805115d3f75d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ldv7" Jul 07 18:55:03.870269 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.870218 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3541e00e-0398-42b2-94f3-805115d3f75d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9ldv7\" (UID: \"3541e00e-0398-42b2-94f3-805115d3f75d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ldv7" Jul 07 18:55:03.870269 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.870236 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8bb75d2e-60c8-4969-ba8d-abfa2cb2a0d1-cert\") pod \"kserve-controller-manager-d6f4b65b8-lnm7t\" (UID: \"8bb75d2e-60c8-4969-ba8d-abfa2cb2a0d1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lnm7t" Jul 07 18:55:03.870416 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:55:03.870355 2564 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 07 18:55:03.870476 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:55:03.870419 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3541e00e-0398-42b2-94f3-805115d3f75d-cert podName:3541e00e-0398-42b2-94f3-805115d3f75d nodeName:}" failed. No retries permitted until 2026-07-07 18:55:04.370403368 +0000 UTC m=+311.789763570 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/3541e00e-0398-42b2-94f3-805115d3f75d-cert") pod "llmisvc-controller-manager-777c6bd7d5-9ldv7" (UID: "3541e00e-0398-42b2-94f3-805115d3f75d") : secret "llmisvc-webhook-server-cert" not found Jul 07 18:55:03.872637 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.872614 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8bb75d2e-60c8-4969-ba8d-abfa2cb2a0d1-cert\") pod \"kserve-controller-manager-d6f4b65b8-lnm7t\" (UID: \"8bb75d2e-60c8-4969-ba8d-abfa2cb2a0d1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lnm7t" Jul 07 18:55:03.883058 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.883034 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgnh6\" (UniqueName: \"kubernetes.io/projected/3541e00e-0398-42b2-94f3-805115d3f75d-kube-api-access-cgnh6\") pod \"llmisvc-controller-manager-777c6bd7d5-9ldv7\" (UID: \"3541e00e-0398-42b2-94f3-805115d3f75d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ldv7" Jul 07 18:55:03.886412 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.886395 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7tkn\" (UniqueName: \"kubernetes.io/projected/8bb75d2e-60c8-4969-ba8d-abfa2cb2a0d1-kube-api-access-p7tkn\") pod \"kserve-controller-manager-d6f4b65b8-lnm7t\" (UID: \"8bb75d2e-60c8-4969-ba8d-abfa2cb2a0d1\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lnm7t" Jul 07 18:55:03.995544 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:03.995494 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lnm7t" Jul 07 18:55:04.359348 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:04.359323 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lnm7t"] Jul 07 18:55:04.359665 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:55:04.359646 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bb75d2e_60c8_4969_ba8d_abfa2cb2a0d1.slice/crio-eb15d1983171a5b3ef26a4349b51b0f594dd5063751457e01502bd364fe11fa0 WatchSource:0}: Error finding container eb15d1983171a5b3ef26a4349b51b0f594dd5063751457e01502bd364fe11fa0: Status 404 returned error can't find the container with id eb15d1983171a5b3ef26a4349b51b0f594dd5063751457e01502bd364fe11fa0 Jul 07 18:55:04.375221 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:04.375196 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3541e00e-0398-42b2-94f3-805115d3f75d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9ldv7\" (UID: \"3541e00e-0398-42b2-94f3-805115d3f75d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ldv7" Jul 07 18:55:04.377564 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:04.377541 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3541e00e-0398-42b2-94f3-805115d3f75d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9ldv7\" (UID: \"3541e00e-0398-42b2-94f3-805115d3f75d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ldv7" Jul 07 18:55:04.556046 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:04.556017 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lnm7t" event={"ID":"8bb75d2e-60c8-4969-ba8d-abfa2cb2a0d1","Type":"ContainerStarted","Data":"eb15d1983171a5b3ef26a4349b51b0f594dd5063751457e01502bd364fe11fa0"} Jul 07 18:55:04.601300 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:04.601276 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ldv7" Jul 07 18:55:05.036197 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:05.036168 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9ldv7"] Jul 07 18:55:05.060424 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:55:05.060391 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3541e00e_0398_42b2_94f3_805115d3f75d.slice/crio-d3d30a391f879dfe088e968904e69a4659546fd30cf383bbdbae102d9aa68865 WatchSource:0}: Error finding container d3d30a391f879dfe088e968904e69a4659546fd30cf383bbdbae102d9aa68865: Status 404 returned error can't find the container with id d3d30a391f879dfe088e968904e69a4659546fd30cf383bbdbae102d9aa68865 Jul 07 18:55:05.562200 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:05.562155 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ldv7" event={"ID":"3541e00e-0398-42b2-94f3-805115d3f75d","Type":"ContainerStarted","Data":"d3d30a391f879dfe088e968904e69a4659546fd30cf383bbdbae102d9aa68865"} Jul 07 18:55:06.125536 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:06.125501 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5"] Jul 07 18:55:06.126004 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:06.125772 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" podUID="14317d37-21eb-4675-8e28-7d9f477bd768" containerName="discovery" containerID="cri-o://6c6134ef71bdb942c077f51c1cf7e301cacf16420e59228cb036dfefb551210b" gracePeriod=30 Jul 07 18:55:06.591286 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:06.588856 2564 generic.go:356] "Generic (PLEG): container finished" podID="14317d37-21eb-4675-8e28-7d9f477bd768" containerID="6c6134ef71bdb942c077f51c1cf7e301cacf16420e59228cb036dfefb551210b" exitCode=0 Jul 07 18:55:06.591286 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:06.588994 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" event={"ID":"14317d37-21eb-4675-8e28-7d9f477bd768","Type":"ContainerDied","Data":"6c6134ef71bdb942c077f51c1cf7e301cacf16420e59228cb036dfefb551210b"} Jul 07 18:55:06.877350 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:06.877327 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:55:07.001021 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.000990 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/14317d37-21eb-4675-8e28-7d9f477bd768-istio-csr-ca-configmap\") pod \"14317d37-21eb-4675-8e28-7d9f477bd768\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " Jul 07 18:55:07.001212 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.001029 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/14317d37-21eb-4675-8e28-7d9f477bd768-istio-csr-dns-cert\") pod \"14317d37-21eb-4675-8e28-7d9f477bd768\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " Jul 07 18:55:07.001212 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.001072 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/14317d37-21eb-4675-8e28-7d9f477bd768-cacerts\") pod \"14317d37-21eb-4675-8e28-7d9f477bd768\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " Jul 07 18:55:07.001212 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.001105 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pmnr\" (UniqueName: \"kubernetes.io/projected/14317d37-21eb-4675-8e28-7d9f477bd768-kube-api-access-9pmnr\") pod \"14317d37-21eb-4675-8e28-7d9f477bd768\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " Jul 07 18:55:07.001212 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.001133 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14317d37-21eb-4675-8e28-7d9f477bd768-istio-kubeconfig\") pod \"14317d37-21eb-4675-8e28-7d9f477bd768\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " Jul 07 18:55:07.001212 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.001173 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14317d37-21eb-4675-8e28-7d9f477bd768-istio-token\") pod \"14317d37-21eb-4675-8e28-7d9f477bd768\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " Jul 07 18:55:07.001459 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.001243 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/14317d37-21eb-4675-8e28-7d9f477bd768-local-certs\") pod \"14317d37-21eb-4675-8e28-7d9f477bd768\" (UID: \"14317d37-21eb-4675-8e28-7d9f477bd768\") " Jul 07 18:55:07.001509 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.001473 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14317d37-21eb-4675-8e28-7d9f477bd768-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "14317d37-21eb-4675-8e28-7d9f477bd768" (UID: "14317d37-21eb-4675-8e28-7d9f477bd768"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:55:07.004066 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.004032 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14317d37-21eb-4675-8e28-7d9f477bd768-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "14317d37-21eb-4675-8e28-7d9f477bd768" (UID: "14317d37-21eb-4675-8e28-7d9f477bd768"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:55:07.004213 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.004195 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14317d37-21eb-4675-8e28-7d9f477bd768-kube-api-access-9pmnr" (OuterVolumeSpecName: "kube-api-access-9pmnr") pod "14317d37-21eb-4675-8e28-7d9f477bd768" (UID: "14317d37-21eb-4675-8e28-7d9f477bd768"). InnerVolumeSpecName "kube-api-access-9pmnr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:55:07.004712 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.004677 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14317d37-21eb-4675-8e28-7d9f477bd768-cacerts" (OuterVolumeSpecName: "cacerts") pod "14317d37-21eb-4675-8e28-7d9f477bd768" (UID: "14317d37-21eb-4675-8e28-7d9f477bd768"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:55:07.004839 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.004754 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14317d37-21eb-4675-8e28-7d9f477bd768-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "14317d37-21eb-4675-8e28-7d9f477bd768" (UID: "14317d37-21eb-4675-8e28-7d9f477bd768"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:55:07.004992 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.004967 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14317d37-21eb-4675-8e28-7d9f477bd768-istio-token" (OuterVolumeSpecName: "istio-token") pod "14317d37-21eb-4675-8e28-7d9f477bd768" (UID: "14317d37-21eb-4675-8e28-7d9f477bd768"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:55:07.005675 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.005649 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14317d37-21eb-4675-8e28-7d9f477bd768-local-certs" (OuterVolumeSpecName: "local-certs") pod "14317d37-21eb-4675-8e28-7d9f477bd768" (UID: "14317d37-21eb-4675-8e28-7d9f477bd768"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:55:07.104134 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.104098 2564 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/14317d37-21eb-4675-8e28-7d9f477bd768-local-certs\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:55:07.104315 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.104139 2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/14317d37-21eb-4675-8e28-7d9f477bd768-istio-csr-ca-configmap\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:55:07.104315 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.104167 2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/14317d37-21eb-4675-8e28-7d9f477bd768-istio-csr-dns-cert\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:55:07.104315 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.104182 2564 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/14317d37-21eb-4675-8e28-7d9f477bd768-cacerts\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:55:07.104315 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.104198 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pmnr\" (UniqueName: \"kubernetes.io/projected/14317d37-21eb-4675-8e28-7d9f477bd768-kube-api-access-9pmnr\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:55:07.104315 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.104214 2564 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/14317d37-21eb-4675-8e28-7d9f477bd768-istio-kubeconfig\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:55:07.104315 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.104234 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14317d37-21eb-4675-8e28-7d9f477bd768-istio-token\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:55:07.595569 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.595535 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lnm7t" event={"ID":"8bb75d2e-60c8-4969-ba8d-abfa2cb2a0d1","Type":"ContainerStarted","Data":"d41c11d7f1e764b3a4e906ddcf52e970ba07f6de1d3eae565b4c04dcb750c20d"} Jul 07 18:55:07.595906 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.595625 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lnm7t" Jul 07 18:55:07.596953 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.596930 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" event={"ID":"14317d37-21eb-4675-8e28-7d9f477bd768","Type":"ContainerDied","Data":"6bbedc706c72ec3d34591a6b06c6750c1db8d43fe6511f0f462621d771b02536"} Jul 07 18:55:07.597054 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.596954 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5" Jul 07 18:55:07.597054 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.596974 2564 scope.go:117] "RemoveContainer" containerID="6c6134ef71bdb942c077f51c1cf7e301cacf16420e59228cb036dfefb551210b" Jul 07 18:55:07.624524 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.624481 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lnm7t" podStartSLOduration=2.246722179 podStartE2EDuration="4.624466782s" totalImagesPullingTime="2.377744603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:55:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:55:07.622672124 +0000 UTC m=+315.042032348" watchObservedRunningTime="2026-07-07 18:55:07.624466782 +0000 UTC m=+315.043827006" Jul 07 18:55:07.649663 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.649635 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5"] Jul 07 18:55:07.653102 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:07.653082 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-zm2n5"] Jul 07 18:55:08.529614 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:08.529559 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-ttzl8" Jul 07 18:55:08.531355 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:08.531336 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-kx4hr" Jul 07 18:55:08.605819 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:08.605789 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ldv7" event={"ID":"3541e00e-0398-42b2-94f3-805115d3f75d","Type":"ContainerStarted","Data":"e8a236ae256a40a248599faa23245919ebd4d24eb32cef189bc87c759ff743a4"} Jul 07 18:55:08.606280 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:08.605949 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ldv7" Jul 07 18:55:08.653480 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:08.653436 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ldv7" podStartSLOduration=3.185557654 podStartE2EDuration="5.653421157s" totalImagesPullingTime="2.467863503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:55:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:55:08.652954026 +0000 UTC m=+316.072314250" watchObservedRunningTime="2026-07-07 18:55:08.653421157 +0000 UTC m=+316.072781439" Jul 07 18:55:09.104059 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:09.104025 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14317d37-21eb-4675-8e28-7d9f477bd768" path="/var/lib/kubelet/pods/14317d37-21eb-4675-8e28-7d9f477bd768/volumes" Jul 07 18:55:38.610888 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:38.610859 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lnm7t" Jul 07 18:55:39.613399 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:39.613365 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ldv7" Jul 07 18:55:52.472157 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:52.472079 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jmchc"] Jul 07 18:55:52.472565 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:52.472437 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14317d37-21eb-4675-8e28-7d9f477bd768" containerName="discovery" Jul 07 18:55:52.472565 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:52.472448 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="14317d37-21eb-4675-8e28-7d9f477bd768" containerName="discovery" Jul 07 18:55:52.472565 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:52.472501 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="14317d37-21eb-4675-8e28-7d9f477bd768" containerName="discovery" Jul 07 18:55:52.474610 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:52.474564 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jmchc" Jul 07 18:55:52.482785 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:52.482762 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 18:55:52.483693 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:52.483667 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 18:55:52.483803 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:52.483729 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-vmqhn\"" Jul 07 18:55:52.493509 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:52.493477 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jmchc"] Jul 07 18:55:52.581973 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:52.581935 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkjbs\" (UniqueName: \"kubernetes.io/projected/b2158a16-053c-4b8c-90c7-488d4202faea-kube-api-access-bkjbs\") pod \"kuadrant-operator-catalog-jmchc\" (UID: \"b2158a16-053c-4b8c-90c7-488d4202faea\") " pod="kuadrant-system/kuadrant-operator-catalog-jmchc" Jul 07 18:55:52.682534 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:52.682499 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkjbs\" (UniqueName: \"kubernetes.io/projected/b2158a16-053c-4b8c-90c7-488d4202faea-kube-api-access-bkjbs\") pod \"kuadrant-operator-catalog-jmchc\" (UID: \"b2158a16-053c-4b8c-90c7-488d4202faea\") " pod="kuadrant-system/kuadrant-operator-catalog-jmchc" Jul 07 18:55:52.691778 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:52.691747 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkjbs\" (UniqueName: \"kubernetes.io/projected/b2158a16-053c-4b8c-90c7-488d4202faea-kube-api-access-bkjbs\") pod \"kuadrant-operator-catalog-jmchc\" (UID: \"b2158a16-053c-4b8c-90c7-488d4202faea\") " pod="kuadrant-system/kuadrant-operator-catalog-jmchc" Jul 07 18:55:52.785016 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:52.784987 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jmchc" Jul 07 18:55:52.815541 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:52.815507 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jmchc"] Jul 07 18:55:53.028346 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:53.028322 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-drrrt"] Jul 07 18:55:53.031545 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:53.031527 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-drrrt" Jul 07 18:55:53.041362 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:53.041339 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-drrrt"] Jul 07 18:55:53.086315 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:53.086287 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqlnk\" (UniqueName: \"kubernetes.io/projected/57988adf-5c24-43ce-8c48-9fa1d0a66f43-kube-api-access-hqlnk\") pod \"kuadrant-operator-catalog-drrrt\" (UID: \"57988adf-5c24-43ce-8c48-9fa1d0a66f43\") " pod="kuadrant-system/kuadrant-operator-catalog-drrrt" Jul 07 18:55:53.153655 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:53.153631 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jmchc"] Jul 07 18:55:53.156031 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:55:53.156006 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2158a16_053c_4b8c_90c7_488d4202faea.slice/crio-97c962e836d4dfcbf60fddc0a8f3e7895de2a74f1606deca1c000063c17a9ec3 WatchSource:0}: Error finding container 97c962e836d4dfcbf60fddc0a8f3e7895de2a74f1606deca1c000063c17a9ec3: Status 404 returned error can't find the container with id 97c962e836d4dfcbf60fddc0a8f3e7895de2a74f1606deca1c000063c17a9ec3 Jul 07 18:55:53.187326 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:53.187296 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqlnk\" (UniqueName: \"kubernetes.io/projected/57988adf-5c24-43ce-8c48-9fa1d0a66f43-kube-api-access-hqlnk\") pod \"kuadrant-operator-catalog-drrrt\" (UID: \"57988adf-5c24-43ce-8c48-9fa1d0a66f43\") " pod="kuadrant-system/kuadrant-operator-catalog-drrrt" Jul 07 18:55:53.200230 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:53.200194 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqlnk\" (UniqueName: \"kubernetes.io/projected/57988adf-5c24-43ce-8c48-9fa1d0a66f43-kube-api-access-hqlnk\") pod \"kuadrant-operator-catalog-drrrt\" (UID: \"57988adf-5c24-43ce-8c48-9fa1d0a66f43\") " pod="kuadrant-system/kuadrant-operator-catalog-drrrt" Jul 07 18:55:53.341235 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:53.341155 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-drrrt" Jul 07 18:55:53.701600 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:53.701554 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-drrrt"] Jul 07 18:55:53.725841 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:55:53.725807 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57988adf_5c24_43ce_8c48_9fa1d0a66f43.slice/crio-3f7670afc40c40a7a044ef93de55df3f52601d92d341f9a2f323a3ee89df5b57 WatchSource:0}: Error finding container 3f7670afc40c40a7a044ef93de55df3f52601d92d341f9a2f323a3ee89df5b57: Status 404 returned error can't find the container with id 3f7670afc40c40a7a044ef93de55df3f52601d92d341f9a2f323a3ee89df5b57 Jul 07 18:55:53.824278 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:53.824243 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-drrrt" event={"ID":"57988adf-5c24-43ce-8c48-9fa1d0a66f43","Type":"ContainerStarted","Data":"3f7670afc40c40a7a044ef93de55df3f52601d92d341f9a2f323a3ee89df5b57"} Jul 07 18:55:53.825419 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:53.825387 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jmchc" event={"ID":"b2158a16-053c-4b8c-90c7-488d4202faea","Type":"ContainerStarted","Data":"97c962e836d4dfcbf60fddc0a8f3e7895de2a74f1606deca1c000063c17a9ec3"} Jul 07 18:55:55.836844 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:55.836808 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-drrrt" event={"ID":"57988adf-5c24-43ce-8c48-9fa1d0a66f43","Type":"ContainerStarted","Data":"df704c2baa764d8532f25756e8d42f0da1e756f70deadc20d46ed147f11e6e85"} Jul 07 18:55:55.838374 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:55.838331 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jmchc" event={"ID":"b2158a16-053c-4b8c-90c7-488d4202faea","Type":"ContainerStarted","Data":"bd63e62bb6bf416b702945d2d634ed6fd08d4fbc7e64a4edf49d25b68b7ca252"} Jul 07 18:55:55.838520 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:55.838405 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-jmchc" podUID="b2158a16-053c-4b8c-90c7-488d4202faea" containerName="registry-server" containerID="cri-o://bd63e62bb6bf416b702945d2d634ed6fd08d4fbc7e64a4edf49d25b68b7ca252" gracePeriod=2 Jul 07 18:55:55.857002 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:55.856960 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-drrrt" podStartSLOduration=1.689453109 podStartE2EDuration="2.856945338s" totalImagesPullingTime="1.167492229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:55:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:55:55.855782457 +0000 UTC m=+363.275142683" watchObservedRunningTime="2026-07-07 18:55:55.856945338 +0000 UTC m=+363.276305563" Jul 07 18:55:55.875611 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:55.875538 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jmchc" podStartSLOduration=2.139308015 podStartE2EDuration="3.875523744s" totalImagesPullingTime="1.736215729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:55:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:55:55.875289052 +0000 UTC m=+363.294649277" watchObservedRunningTime="2026-07-07 18:55:55.875523744 +0000 UTC m=+363.294883969" Jul 07 18:55:56.305230 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:56.305202 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jmchc" Jul 07 18:55:56.417359 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:56.417326 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bkjbs\" (UniqueName: \"kubernetes.io/projected/b2158a16-053c-4b8c-90c7-488d4202faea-kube-api-access-bkjbs\") pod \"b2158a16-053c-4b8c-90c7-488d4202faea\" (UID: \"b2158a16-053c-4b8c-90c7-488d4202faea\") " Jul 07 18:55:56.419750 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:56.419723 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2158a16-053c-4b8c-90c7-488d4202faea-kube-api-access-bkjbs" (OuterVolumeSpecName: "kube-api-access-bkjbs") pod "b2158a16-053c-4b8c-90c7-488d4202faea" (UID: "b2158a16-053c-4b8c-90c7-488d4202faea"). InnerVolumeSpecName "kube-api-access-bkjbs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:55:56.518204 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:56.518159 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkjbs\" (UniqueName: \"kubernetes.io/projected/b2158a16-053c-4b8c-90c7-488d4202faea-kube-api-access-bkjbs\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:55:56.845254 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:56.845220 2564 generic.go:356] "Generic (PLEG): container finished" podID="b2158a16-053c-4b8c-90c7-488d4202faea" containerID="bd63e62bb6bf416b702945d2d634ed6fd08d4fbc7e64a4edf49d25b68b7ca252" exitCode=0 Jul 07 18:55:56.845696 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:56.845265 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jmchc" Jul 07 18:55:56.845696 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:56.845303 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jmchc" event={"ID":"b2158a16-053c-4b8c-90c7-488d4202faea","Type":"ContainerDied","Data":"bd63e62bb6bf416b702945d2d634ed6fd08d4fbc7e64a4edf49d25b68b7ca252"} Jul 07 18:55:56.845696 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:56.845354 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jmchc" event={"ID":"b2158a16-053c-4b8c-90c7-488d4202faea","Type":"ContainerDied","Data":"97c962e836d4dfcbf60fddc0a8f3e7895de2a74f1606deca1c000063c17a9ec3"} Jul 07 18:55:56.845696 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:56.845374 2564 scope.go:117] "RemoveContainer" containerID="bd63e62bb6bf416b702945d2d634ed6fd08d4fbc7e64a4edf49d25b68b7ca252" Jul 07 18:55:56.855098 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:56.855079 2564 scope.go:117] "RemoveContainer" containerID="bd63e62bb6bf416b702945d2d634ed6fd08d4fbc7e64a4edf49d25b68b7ca252" Jul 07 18:55:56.855395 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:55:56.855376 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd63e62bb6bf416b702945d2d634ed6fd08d4fbc7e64a4edf49d25b68b7ca252\": container with ID starting with bd63e62bb6bf416b702945d2d634ed6fd08d4fbc7e64a4edf49d25b68b7ca252 not found: ID does not exist" containerID="bd63e62bb6bf416b702945d2d634ed6fd08d4fbc7e64a4edf49d25b68b7ca252" Jul 07 18:55:56.855441 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:56.855405 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd63e62bb6bf416b702945d2d634ed6fd08d4fbc7e64a4edf49d25b68b7ca252"} err="failed to get container status \"bd63e62bb6bf416b702945d2d634ed6fd08d4fbc7e64a4edf49d25b68b7ca252\": rpc error: code = NotFound desc = could not find container \"bd63e62bb6bf416b702945d2d634ed6fd08d4fbc7e64a4edf49d25b68b7ca252\": container with ID starting with bd63e62bb6bf416b702945d2d634ed6fd08d4fbc7e64a4edf49d25b68b7ca252 not found: ID does not exist" Jul 07 18:55:56.867284 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:56.867248 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jmchc"] Jul 07 18:55:56.871632 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:56.871602 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jmchc"] Jul 07 18:55:57.103240 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:55:57.103156 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2158a16-053c-4b8c-90c7-488d4202faea" path="/var/lib/kubelet/pods/b2158a16-053c-4b8c-90c7-488d4202faea/volumes" Jul 07 18:56:03.342083 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:03.342042 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-drrrt" Jul 07 18:56:03.342569 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:03.342130 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-drrrt" Jul 07 18:56:03.366208 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:03.366179 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-drrrt" Jul 07 18:56:03.908668 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:03.908641 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-drrrt" Jul 07 18:56:07.856171 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:07.856136 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2"] Jul 07 18:56:07.856550 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:07.856481 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2158a16-053c-4b8c-90c7-488d4202faea" containerName="registry-server" Jul 07 18:56:07.856550 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:07.856492 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2158a16-053c-4b8c-90c7-488d4202faea" containerName="registry-server" Jul 07 18:56:07.856550 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:07.856550 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2158a16-053c-4b8c-90c7-488d4202faea" containerName="registry-server" Jul 07 18:56:07.859528 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:07.859511 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2" Jul 07 18:56:07.861958 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:07.861936 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-862l7\"" Jul 07 18:56:07.870227 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:07.870202 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2"] Jul 07 18:56:07.917310 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:07.917279 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67ee7edd-7a6b-46a6-869e-d2054008e73c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2\" (UID: \"67ee7edd-7a6b-46a6-869e-d2054008e73c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2" Jul 07 18:56:07.917310 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:07.917312 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67ee7edd-7a6b-46a6-869e-d2054008e73c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2\" (UID: \"67ee7edd-7a6b-46a6-869e-d2054008e73c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2" Jul 07 18:56:07.917519 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:07.917348 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9b45\" (UniqueName: \"kubernetes.io/projected/67ee7edd-7a6b-46a6-869e-d2054008e73c-kube-api-access-x9b45\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2\" (UID: \"67ee7edd-7a6b-46a6-869e-d2054008e73c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2" Jul 07 18:56:08.018264 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.018231 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67ee7edd-7a6b-46a6-869e-d2054008e73c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2\" (UID: \"67ee7edd-7a6b-46a6-869e-d2054008e73c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2" Jul 07 18:56:08.018264 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.018268 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67ee7edd-7a6b-46a6-869e-d2054008e73c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2\" (UID: \"67ee7edd-7a6b-46a6-869e-d2054008e73c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2" Jul 07 18:56:08.018521 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.018296 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9b45\" (UniqueName: \"kubernetes.io/projected/67ee7edd-7a6b-46a6-869e-d2054008e73c-kube-api-access-x9b45\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2\" (UID: \"67ee7edd-7a6b-46a6-869e-d2054008e73c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2" Jul 07 18:56:08.018684 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.018662 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67ee7edd-7a6b-46a6-869e-d2054008e73c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2\" (UID: \"67ee7edd-7a6b-46a6-869e-d2054008e73c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2" Jul 07 18:56:08.018761 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.018722 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67ee7edd-7a6b-46a6-869e-d2054008e73c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2\" (UID: \"67ee7edd-7a6b-46a6-869e-d2054008e73c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2" Jul 07 18:56:08.029890 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.029858 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9b45\" (UniqueName: \"kubernetes.io/projected/67ee7edd-7a6b-46a6-869e-d2054008e73c-kube-api-access-x9b45\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2\" (UID: \"67ee7edd-7a6b-46a6-869e-d2054008e73c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2" Jul 07 18:56:08.169752 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.169671 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2" Jul 07 18:56:08.467650 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.467537 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h"] Jul 07 18:56:08.470870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.470850 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h" Jul 07 18:56:08.484782 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.484757 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h"] Jul 07 18:56:08.523566 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.523424 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2"] Jul 07 18:56:08.526511 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:56:08.526485 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67ee7edd_7a6b_46a6_869e_d2054008e73c.slice/crio-15c3a3480a6dceb27da535197c478b42b1506d59a664466c7d07d2815cf5be43 WatchSource:0}: Error finding container 15c3a3480a6dceb27da535197c478b42b1506d59a664466c7d07d2815cf5be43: Status 404 returned error can't find the container with id 15c3a3480a6dceb27da535197c478b42b1506d59a664466c7d07d2815cf5be43 Jul 07 18:56:08.623653 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.623623 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a0969383-1c72-470a-a990-7ef7d6a752d2-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h\" (UID: \"a0969383-1c72-470a-a990-7ef7d6a752d2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h" Jul 07 18:56:08.623781 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.623675 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a0969383-1c72-470a-a990-7ef7d6a752d2-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h\" (UID: \"a0969383-1c72-470a-a990-7ef7d6a752d2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h" Jul 07 18:56:08.623781 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.623766 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcspq\" (UniqueName: \"kubernetes.io/projected/a0969383-1c72-470a-a990-7ef7d6a752d2-kube-api-access-fcspq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h\" (UID: \"a0969383-1c72-470a-a990-7ef7d6a752d2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h" Jul 07 18:56:08.724399 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.724306 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcspq\" (UniqueName: \"kubernetes.io/projected/a0969383-1c72-470a-a990-7ef7d6a752d2-kube-api-access-fcspq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h\" (UID: \"a0969383-1c72-470a-a990-7ef7d6a752d2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h" Jul 07 18:56:08.724399 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.724388 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a0969383-1c72-470a-a990-7ef7d6a752d2-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h\" (UID: \"a0969383-1c72-470a-a990-7ef7d6a752d2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h" Jul 07 18:56:08.724597 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.724416 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a0969383-1c72-470a-a990-7ef7d6a752d2-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h\" (UID: \"a0969383-1c72-470a-a990-7ef7d6a752d2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h" Jul 07 18:56:08.724745 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.724724 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a0969383-1c72-470a-a990-7ef7d6a752d2-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h\" (UID: \"a0969383-1c72-470a-a990-7ef7d6a752d2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h" Jul 07 18:56:08.724782 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.724764 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a0969383-1c72-470a-a990-7ef7d6a752d2-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h\" (UID: \"a0969383-1c72-470a-a990-7ef7d6a752d2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h" Jul 07 18:56:08.733563 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.733538 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcspq\" (UniqueName: \"kubernetes.io/projected/a0969383-1c72-470a-a990-7ef7d6a752d2-kube-api-access-fcspq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h\" (UID: \"a0969383-1c72-470a-a990-7ef7d6a752d2\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h" Jul 07 18:56:08.780225 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.780196 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h" Jul 07 18:56:08.853642 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.853610 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl"] Jul 07 18:56:08.856716 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.856694 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl" Jul 07 18:56:08.871176 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.871143 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl"] Jul 07 18:56:08.910315 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.910281 2564 generic.go:356] "Generic (PLEG): container finished" podID="67ee7edd-7a6b-46a6-869e-d2054008e73c" containerID="084c8460da6b4ae0a1409e3d8edeaa0e1169989a3391bccd2d43b4a7921451ea" exitCode=0 Jul 07 18:56:08.910472 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.910367 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2" event={"ID":"67ee7edd-7a6b-46a6-869e-d2054008e73c","Type":"ContainerDied","Data":"084c8460da6b4ae0a1409e3d8edeaa0e1169989a3391bccd2d43b4a7921451ea"} Jul 07 18:56:08.910472 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.910414 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2" event={"ID":"67ee7edd-7a6b-46a6-869e-d2054008e73c","Type":"ContainerStarted","Data":"15c3a3480a6dceb27da535197c478b42b1506d59a664466c7d07d2815cf5be43"} Jul 07 18:56:08.926239 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.926211 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl\" (UID: \"5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl" Jul 07 18:56:08.926371 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.926244 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4kxl\" (UniqueName: \"kubernetes.io/projected/5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf-kube-api-access-n4kxl\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl\" (UID: \"5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl" Jul 07 18:56:08.926371 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:08.926347 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl\" (UID: \"5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl" Jul 07 18:56:09.027030 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.026995 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl\" (UID: \"5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl" Jul 07 18:56:09.027030 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.027036 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4kxl\" (UniqueName: \"kubernetes.io/projected/5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf-kube-api-access-n4kxl\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl\" (UID: \"5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl" Jul 07 18:56:09.027235 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.027091 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl\" (UID: \"5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl" Jul 07 18:56:09.027399 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.027380 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl\" (UID: \"5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl" Jul 07 18:56:09.027451 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.027410 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl\" (UID: \"5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl" Jul 07 18:56:09.037685 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.037661 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4kxl\" (UniqueName: \"kubernetes.io/projected/5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf-kube-api-access-n4kxl\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl\" (UID: \"5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl" Jul 07 18:56:09.131382 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.131249 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h"] Jul 07 18:56:09.133532 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:56:09.133501 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0969383_1c72_470a_a990_7ef7d6a752d2.slice/crio-fe59d7d08f95aa077119dfb4e1a5b55c117190f77672ec0776bce1979b7b1d22 WatchSource:0}: Error finding container fe59d7d08f95aa077119dfb4e1a5b55c117190f77672ec0776bce1979b7b1d22: Status 404 returned error can't find the container with id fe59d7d08f95aa077119dfb4e1a5b55c117190f77672ec0776bce1979b7b1d22 Jul 07 18:56:09.166357 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.166327 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl" Jul 07 18:56:09.257052 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.256650 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd"] Jul 07 18:56:09.259478 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.259455 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd" Jul 07 18:56:09.272234 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.272204 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd"] Jul 07 18:56:09.329867 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.329794 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65401510-654a-4e3c-b89b-809ee03e398c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd\" (UID: \"65401510-654a-4e3c-b89b-809ee03e398c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd" Jul 07 18:56:09.329867 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.329842 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l24fn\" (UniqueName: \"kubernetes.io/projected/65401510-654a-4e3c-b89b-809ee03e398c-kube-api-access-l24fn\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd\" (UID: \"65401510-654a-4e3c-b89b-809ee03e398c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd" Jul 07 18:56:09.329867 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.329863 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65401510-654a-4e3c-b89b-809ee03e398c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd\" (UID: \"65401510-654a-4e3c-b89b-809ee03e398c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd" Jul 07 18:56:09.431015 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.430986 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65401510-654a-4e3c-b89b-809ee03e398c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd\" (UID: \"65401510-654a-4e3c-b89b-809ee03e398c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd" Jul 07 18:56:09.431184 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.431029 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l24fn\" (UniqueName: \"kubernetes.io/projected/65401510-654a-4e3c-b89b-809ee03e398c-kube-api-access-l24fn\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd\" (UID: \"65401510-654a-4e3c-b89b-809ee03e398c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd" Jul 07 18:56:09.431184 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.431048 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65401510-654a-4e3c-b89b-809ee03e398c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd\" (UID: \"65401510-654a-4e3c-b89b-809ee03e398c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd" Jul 07 18:56:09.431326 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.431310 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65401510-654a-4e3c-b89b-809ee03e398c-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd\" (UID: \"65401510-654a-4e3c-b89b-809ee03e398c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd" Jul 07 18:56:09.431417 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.431398 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65401510-654a-4e3c-b89b-809ee03e398c-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd\" (UID: \"65401510-654a-4e3c-b89b-809ee03e398c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd" Jul 07 18:56:09.440397 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.440376 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l24fn\" (UniqueName: \"kubernetes.io/projected/65401510-654a-4e3c-b89b-809ee03e398c-kube-api-access-l24fn\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd\" (UID: \"65401510-654a-4e3c-b89b-809ee03e398c\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd" Jul 07 18:56:09.522665 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.522641 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl"] Jul 07 18:56:09.531625 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:56:09.531597 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ad39d3b_c5ce_439e_9fca_4eeaf48a3fbf.slice/crio-33c9f9b4784a949a3dc5972461d2783cf4b2bd7576bba983b136b82098a6ee89 WatchSource:0}: Error finding container 33c9f9b4784a949a3dc5972461d2783cf4b2bd7576bba983b136b82098a6ee89: Status 404 returned error can't find the container with id 33c9f9b4784a949a3dc5972461d2783cf4b2bd7576bba983b136b82098a6ee89 Jul 07 18:56:09.685743 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.685711 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd" Jul 07 18:56:09.916775 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.916696 2564 generic.go:356] "Generic (PLEG): container finished" podID="5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf" containerID="a54492e9c7e013ecc8dc352ecc71811a7929c191450ce59540642951748a0b3f" exitCode=0 Jul 07 18:56:09.917237 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.916778 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl" event={"ID":"5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf","Type":"ContainerDied","Data":"a54492e9c7e013ecc8dc352ecc71811a7929c191450ce59540642951748a0b3f"} Jul 07 18:56:09.917237 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.916832 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl" event={"ID":"5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf","Type":"ContainerStarted","Data":"33c9f9b4784a949a3dc5972461d2783cf4b2bd7576bba983b136b82098a6ee89"} Jul 07 18:56:09.918317 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.918296 2564 generic.go:356] "Generic (PLEG): container finished" podID="a0969383-1c72-470a-a990-7ef7d6a752d2" containerID="c4c2532595eff3de8260f72805022a4b1b642860bea20d6e2017f20026446117" exitCode=0 Jul 07 18:56:09.918434 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.918325 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h" event={"ID":"a0969383-1c72-470a-a990-7ef7d6a752d2","Type":"ContainerDied","Data":"c4c2532595eff3de8260f72805022a4b1b642860bea20d6e2017f20026446117"} Jul 07 18:56:09.918434 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.918362 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h" event={"ID":"a0969383-1c72-470a-a990-7ef7d6a752d2","Type":"ContainerStarted","Data":"fe59d7d08f95aa077119dfb4e1a5b55c117190f77672ec0776bce1979b7b1d22"} Jul 07 18:56:09.920330 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.920310 2564 generic.go:356] "Generic (PLEG): container finished" podID="67ee7edd-7a6b-46a6-869e-d2054008e73c" containerID="dc4cfd57b21a6259d9bd0402a00567b0ef210154c8bdf9d66f7ca7d1b09e762f" exitCode=0 Jul 07 18:56:09.920430 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:09.920373 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2" event={"ID":"67ee7edd-7a6b-46a6-869e-d2054008e73c","Type":"ContainerDied","Data":"dc4cfd57b21a6259d9bd0402a00567b0ef210154c8bdf9d66f7ca7d1b09e762f"} Jul 07 18:56:10.028007 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:10.027911 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd"] Jul 07 18:56:10.030866 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:56:10.030837 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65401510_654a_4e3c_b89b_809ee03e398c.slice/crio-683fab27b26d2fcf185913afda19aaeb0c4406485c949bac1c87cbc23e5063b7 WatchSource:0}: Error finding container 683fab27b26d2fcf185913afda19aaeb0c4406485c949bac1c87cbc23e5063b7: Status 404 returned error can't find the container with id 683fab27b26d2fcf185913afda19aaeb0c4406485c949bac1c87cbc23e5063b7 Jul 07 18:56:10.926844 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:10.926819 2564 generic.go:356] "Generic (PLEG): container finished" podID="67ee7edd-7a6b-46a6-869e-d2054008e73c" containerID="870dbbb93008f7d0956da14b16f5f3fb0e125d0398b780ca287663b028410595" exitCode=0 Jul 07 18:56:10.927281 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:10.926897 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2" event={"ID":"67ee7edd-7a6b-46a6-869e-d2054008e73c","Type":"ContainerDied","Data":"870dbbb93008f7d0956da14b16f5f3fb0e125d0398b780ca287663b028410595"} Jul 07 18:56:10.934977 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:10.934949 2564 generic.go:356] "Generic (PLEG): container finished" podID="5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf" containerID="070f0399176a9268d8ec6a7fff70d8c1ca01718dd31e05072a00100fb1788dbf" exitCode=0 Jul 07 18:56:10.935103 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:10.935025 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl" event={"ID":"5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf","Type":"ContainerDied","Data":"070f0399176a9268d8ec6a7fff70d8c1ca01718dd31e05072a00100fb1788dbf"} Jul 07 18:56:10.936843 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:10.936819 2564 generic.go:356] "Generic (PLEG): container finished" podID="a0969383-1c72-470a-a990-7ef7d6a752d2" containerID="9231651ff86a4e242b668db32aac188d780df8a06c04a548a12d9c356b1e5e99" exitCode=0 Jul 07 18:56:10.936946 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:10.936904 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h" event={"ID":"a0969383-1c72-470a-a990-7ef7d6a752d2","Type":"ContainerDied","Data":"9231651ff86a4e242b668db32aac188d780df8a06c04a548a12d9c356b1e5e99"} Jul 07 18:56:10.938372 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:10.938354 2564 generic.go:356] "Generic (PLEG): container finished" podID="65401510-654a-4e3c-b89b-809ee03e398c" containerID="9903d2ec634a6b5e6ec40f0eaecd28d3f21953947b8a8658d6019392f186ff43" exitCode=0 Jul 07 18:56:10.938472 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:10.938395 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd" event={"ID":"65401510-654a-4e3c-b89b-809ee03e398c","Type":"ContainerDied","Data":"9903d2ec634a6b5e6ec40f0eaecd28d3f21953947b8a8658d6019392f186ff43"} Jul 07 18:56:10.938472 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:10.938429 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd" event={"ID":"65401510-654a-4e3c-b89b-809ee03e398c","Type":"ContainerStarted","Data":"683fab27b26d2fcf185913afda19aaeb0c4406485c949bac1c87cbc23e5063b7"} Jul 07 18:56:11.946186 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:11.946095 2564 generic.go:356] "Generic (PLEG): container finished" podID="5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf" containerID="f1eb1e32b8c3672737613d4a2fbc3b70443cbfdd4e671e709747f185b79f034f" exitCode=0 Jul 07 18:56:11.946186 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:11.946150 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl" event={"ID":"5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf","Type":"ContainerDied","Data":"f1eb1e32b8c3672737613d4a2fbc3b70443cbfdd4e671e709747f185b79f034f"} Jul 07 18:56:11.948269 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:11.948245 2564 generic.go:356] "Generic (PLEG): container finished" podID="a0969383-1c72-470a-a990-7ef7d6a752d2" containerID="b4bed82813395b60396b1cc59e7f8412acda73161be8174ff2fce799a49cf395" exitCode=0 Jul 07 18:56:11.948428 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:11.948321 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h" event={"ID":"a0969383-1c72-470a-a990-7ef7d6a752d2","Type":"ContainerDied","Data":"b4bed82813395b60396b1cc59e7f8412acda73161be8174ff2fce799a49cf395"} Jul 07 18:56:11.949997 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:11.949980 2564 generic.go:356] "Generic (PLEG): container finished" podID="65401510-654a-4e3c-b89b-809ee03e398c" containerID="cd82f2f725f64b697f21e12f1515a8e616f8eea8b410ce2a1a6dc2bbcea3bd86" exitCode=0 Jul 07 18:56:11.950138 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:11.950111 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd" event={"ID":"65401510-654a-4e3c-b89b-809ee03e398c","Type":"ContainerDied","Data":"cd82f2f725f64b697f21e12f1515a8e616f8eea8b410ce2a1a6dc2bbcea3bd86"} Jul 07 18:56:12.301192 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:12.301163 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2" Jul 07 18:56:12.454754 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:12.454724 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67ee7edd-7a6b-46a6-869e-d2054008e73c-bundle\") pod \"67ee7edd-7a6b-46a6-869e-d2054008e73c\" (UID: \"67ee7edd-7a6b-46a6-869e-d2054008e73c\") " Jul 07 18:56:12.454920 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:12.454805 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67ee7edd-7a6b-46a6-869e-d2054008e73c-util\") pod \"67ee7edd-7a6b-46a6-869e-d2054008e73c\" (UID: \"67ee7edd-7a6b-46a6-869e-d2054008e73c\") " Jul 07 18:56:12.454920 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:12.454838 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9b45\" (UniqueName: \"kubernetes.io/projected/67ee7edd-7a6b-46a6-869e-d2054008e73c-kube-api-access-x9b45\") pod \"67ee7edd-7a6b-46a6-869e-d2054008e73c\" (UID: \"67ee7edd-7a6b-46a6-869e-d2054008e73c\") " Jul 07 18:56:12.455207 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:12.455172 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67ee7edd-7a6b-46a6-869e-d2054008e73c-bundle" (OuterVolumeSpecName: "bundle") pod "67ee7edd-7a6b-46a6-869e-d2054008e73c" (UID: "67ee7edd-7a6b-46a6-869e-d2054008e73c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:56:12.457201 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:12.457174 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67ee7edd-7a6b-46a6-869e-d2054008e73c-kube-api-access-x9b45" (OuterVolumeSpecName: "kube-api-access-x9b45") pod "67ee7edd-7a6b-46a6-869e-d2054008e73c" (UID: "67ee7edd-7a6b-46a6-869e-d2054008e73c"). InnerVolumeSpecName "kube-api-access-x9b45". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:56:12.460352 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:12.460332 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67ee7edd-7a6b-46a6-869e-d2054008e73c-util" (OuterVolumeSpecName: "util") pod "67ee7edd-7a6b-46a6-869e-d2054008e73c" (UID: "67ee7edd-7a6b-46a6-869e-d2054008e73c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:56:12.555554 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:12.555463 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67ee7edd-7a6b-46a6-869e-d2054008e73c-util\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:56:12.555554 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:12.555486 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9b45\" (UniqueName: \"kubernetes.io/projected/67ee7edd-7a6b-46a6-869e-d2054008e73c-kube-api-access-x9b45\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:56:12.555554 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:12.555496 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67ee7edd-7a6b-46a6-869e-d2054008e73c-bundle\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:56:12.957122 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:12.957044 2564 generic.go:356] "Generic (PLEG): container finished" podID="65401510-654a-4e3c-b89b-809ee03e398c" containerID="a4dafd512fa364376da7c749987b089f6ec05be280ad5a82b1f4aa0c43251abf" exitCode=0 Jul 07 18:56:12.957500 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:12.957122 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd" event={"ID":"65401510-654a-4e3c-b89b-809ee03e398c","Type":"ContainerDied","Data":"a4dafd512fa364376da7c749987b089f6ec05be280ad5a82b1f4aa0c43251abf"} Jul 07 18:56:12.958987 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:12.958969 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2" event={"ID":"67ee7edd-7a6b-46a6-869e-d2054008e73c","Type":"ContainerDied","Data":"15c3a3480a6dceb27da535197c478b42b1506d59a664466c7d07d2815cf5be43"} Jul 07 18:56:12.959085 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:12.958999 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2" Jul 07 18:56:12.959085 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:12.959000 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="15c3a3480a6dceb27da535197c478b42b1506d59a664466c7d07d2815cf5be43" Jul 07 18:56:13.337891 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.337869 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl" Jul 07 18:56:13.466626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.466602 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf-util\") pod \"5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf\" (UID: \"5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf\") " Jul 07 18:56:13.466748 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.466681 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4kxl\" (UniqueName: \"kubernetes.io/projected/5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf-kube-api-access-n4kxl\") pod \"5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf\" (UID: \"5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf\") " Jul 07 18:56:13.466820 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.466784 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf-bundle\") pod \"5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf\" (UID: \"5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf\") " Jul 07 18:56:13.467349 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.467322 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf-bundle" (OuterVolumeSpecName: "bundle") pod "5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf" (UID: "5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:56:13.468892 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.468868 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf-kube-api-access-n4kxl" (OuterVolumeSpecName: "kube-api-access-n4kxl") pod "5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf" (UID: "5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf"). InnerVolumeSpecName "kube-api-access-n4kxl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:56:13.471777 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.471755 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h" Jul 07 18:56:13.472810 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.472789 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf-util" (OuterVolumeSpecName: "util") pod "5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf" (UID: "5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:56:13.567549 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.567521 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a0969383-1c72-470a-a990-7ef7d6a752d2-bundle\") pod \"a0969383-1c72-470a-a990-7ef7d6a752d2\" (UID: \"a0969383-1c72-470a-a990-7ef7d6a752d2\") " Jul 07 18:56:13.567709 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.567610 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a0969383-1c72-470a-a990-7ef7d6a752d2-util\") pod \"a0969383-1c72-470a-a990-7ef7d6a752d2\" (UID: \"a0969383-1c72-470a-a990-7ef7d6a752d2\") " Jul 07 18:56:13.567709 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.567633 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcspq\" (UniqueName: \"kubernetes.io/projected/a0969383-1c72-470a-a990-7ef7d6a752d2-kube-api-access-fcspq\") pod \"a0969383-1c72-470a-a990-7ef7d6a752d2\" (UID: \"a0969383-1c72-470a-a990-7ef7d6a752d2\") " Jul 07 18:56:13.567884 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.567867 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf-bundle\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:56:13.567948 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.567891 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf-util\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:56:13.567948 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.567907 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4kxl\" (UniqueName: \"kubernetes.io/projected/5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf-kube-api-access-n4kxl\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:56:13.568107 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.568080 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0969383-1c72-470a-a990-7ef7d6a752d2-bundle" (OuterVolumeSpecName: "bundle") pod "a0969383-1c72-470a-a990-7ef7d6a752d2" (UID: "a0969383-1c72-470a-a990-7ef7d6a752d2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:56:13.569871 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.569846 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0969383-1c72-470a-a990-7ef7d6a752d2-kube-api-access-fcspq" (OuterVolumeSpecName: "kube-api-access-fcspq") pod "a0969383-1c72-470a-a990-7ef7d6a752d2" (UID: "a0969383-1c72-470a-a990-7ef7d6a752d2"). InnerVolumeSpecName "kube-api-access-fcspq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:56:13.573159 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.573130 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0969383-1c72-470a-a990-7ef7d6a752d2-util" (OuterVolumeSpecName: "util") pod "a0969383-1c72-470a-a990-7ef7d6a752d2" (UID: "a0969383-1c72-470a-a990-7ef7d6a752d2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:56:13.668453 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.668427 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a0969383-1c72-470a-a990-7ef7d6a752d2-bundle\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:56:13.668453 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.668449 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a0969383-1c72-470a-a990-7ef7d6a752d2-util\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:56:13.668599 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.668459 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcspq\" (UniqueName: \"kubernetes.io/projected/a0969383-1c72-470a-a990-7ef7d6a752d2-kube-api-access-fcspq\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:56:13.968109 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.968046 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl" Jul 07 18:56:13.968478 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.968040 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl" event={"ID":"5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf","Type":"ContainerDied","Data":"33c9f9b4784a949a3dc5972461d2783cf4b2bd7576bba983b136b82098a6ee89"} Jul 07 18:56:13.968478 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.968200 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="33c9f9b4784a949a3dc5972461d2783cf4b2bd7576bba983b136b82098a6ee89" Jul 07 18:56:13.969935 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.969915 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h" Jul 07 18:56:13.970054 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.969948 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h" event={"ID":"a0969383-1c72-470a-a990-7ef7d6a752d2","Type":"ContainerDied","Data":"fe59d7d08f95aa077119dfb4e1a5b55c117190f77672ec0776bce1979b7b1d22"} Jul 07 18:56:13.970054 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:13.969990 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fe59d7d08f95aa077119dfb4e1a5b55c117190f77672ec0776bce1979b7b1d22" Jul 07 18:56:14.300285 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:14.300257 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd" Jul 07 18:56:14.374441 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:14.374410 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l24fn\" (UniqueName: \"kubernetes.io/projected/65401510-654a-4e3c-b89b-809ee03e398c-kube-api-access-l24fn\") pod \"65401510-654a-4e3c-b89b-809ee03e398c\" (UID: \"65401510-654a-4e3c-b89b-809ee03e398c\") " Jul 07 18:56:14.374636 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:14.374451 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65401510-654a-4e3c-b89b-809ee03e398c-util\") pod \"65401510-654a-4e3c-b89b-809ee03e398c\" (UID: \"65401510-654a-4e3c-b89b-809ee03e398c\") " Jul 07 18:56:14.374636 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:14.374478 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65401510-654a-4e3c-b89b-809ee03e398c-bundle\") pod \"65401510-654a-4e3c-b89b-809ee03e398c\" (UID: \"65401510-654a-4e3c-b89b-809ee03e398c\") " Jul 07 18:56:14.374978 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:14.374952 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65401510-654a-4e3c-b89b-809ee03e398c-bundle" (OuterVolumeSpecName: "bundle") pod "65401510-654a-4e3c-b89b-809ee03e398c" (UID: "65401510-654a-4e3c-b89b-809ee03e398c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:56:14.376628 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:14.376600 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65401510-654a-4e3c-b89b-809ee03e398c-kube-api-access-l24fn" (OuterVolumeSpecName: "kube-api-access-l24fn") pod "65401510-654a-4e3c-b89b-809ee03e398c" (UID: "65401510-654a-4e3c-b89b-809ee03e398c"). InnerVolumeSpecName "kube-api-access-l24fn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:56:14.379838 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:14.379817 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65401510-654a-4e3c-b89b-809ee03e398c-util" (OuterVolumeSpecName: "util") pod "65401510-654a-4e3c-b89b-809ee03e398c" (UID: "65401510-654a-4e3c-b89b-809ee03e398c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:56:14.475644 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:14.475618 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l24fn\" (UniqueName: \"kubernetes.io/projected/65401510-654a-4e3c-b89b-809ee03e398c-kube-api-access-l24fn\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:56:14.475644 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:14.475640 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/65401510-654a-4e3c-b89b-809ee03e398c-util\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:56:14.475644 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:14.475649 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/65401510-654a-4e3c-b89b-809ee03e398c-bundle\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:56:14.976967 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:14.976945 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd" Jul 07 18:56:14.976967 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:14.976960 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd" event={"ID":"65401510-654a-4e3c-b89b-809ee03e398c","Type":"ContainerDied","Data":"683fab27b26d2fcf185913afda19aaeb0c4406485c949bac1c87cbc23e5063b7"} Jul 07 18:56:14.977363 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:14.976999 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="683fab27b26d2fcf185913afda19aaeb0c4406485c949bac1c87cbc23e5063b7" Jul 07 18:56:26.015818 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.015788 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5"] Jul 07 18:56:26.016164 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016089 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67ee7edd-7a6b-46a6-869e-d2054008e73c" containerName="extract" Jul 07 18:56:26.016164 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016100 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="67ee7edd-7a6b-46a6-869e-d2054008e73c" containerName="extract" Jul 07 18:56:26.016164 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016108 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf" containerName="util" Jul 07 18:56:26.016164 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016113 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf" containerName="util" Jul 07 18:56:26.016164 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016120 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0969383-1c72-470a-a990-7ef7d6a752d2" containerName="pull" Jul 07 18:56:26.016164 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016125 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0969383-1c72-470a-a990-7ef7d6a752d2" containerName="pull" Jul 07 18:56:26.016164 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016132 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65401510-654a-4e3c-b89b-809ee03e398c" containerName="pull" Jul 07 18:56:26.016164 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016137 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="65401510-654a-4e3c-b89b-809ee03e398c" containerName="pull" Jul 07 18:56:26.016164 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016144 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67ee7edd-7a6b-46a6-869e-d2054008e73c" containerName="pull" Jul 07 18:56:26.016164 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016149 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="67ee7edd-7a6b-46a6-869e-d2054008e73c" containerName="pull" Jul 07 18:56:26.016164 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016153 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0969383-1c72-470a-a990-7ef7d6a752d2" containerName="extract" Jul 07 18:56:26.016164 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016158 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0969383-1c72-470a-a990-7ef7d6a752d2" containerName="extract" Jul 07 18:56:26.016164 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016164 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65401510-654a-4e3c-b89b-809ee03e398c" containerName="util" Jul 07 18:56:26.016164 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016169 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="65401510-654a-4e3c-b89b-809ee03e398c" containerName="util" Jul 07 18:56:26.016640 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016174 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf" containerName="extract" Jul 07 18:56:26.016640 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016179 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf" containerName="extract" Jul 07 18:56:26.016640 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016186 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67ee7edd-7a6b-46a6-869e-d2054008e73c" containerName="util" Jul 07 18:56:26.016640 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016190 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="67ee7edd-7a6b-46a6-869e-d2054008e73c" containerName="util" Jul 07 18:56:26.016640 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016197 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf" containerName="pull" Jul 07 18:56:26.016640 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016201 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf" containerName="pull" Jul 07 18:56:26.016640 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016207 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0969383-1c72-470a-a990-7ef7d6a752d2" containerName="util" Jul 07 18:56:26.016640 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016211 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0969383-1c72-470a-a990-7ef7d6a752d2" containerName="util" Jul 07 18:56:26.016640 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016218 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65401510-654a-4e3c-b89b-809ee03e398c" containerName="extract" Jul 07 18:56:26.016640 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016222 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="65401510-654a-4e3c-b89b-809ee03e398c" containerName="extract" Jul 07 18:56:26.016640 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016266 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0969383-1c72-470a-a990-7ef7d6a752d2" containerName="extract" Jul 07 18:56:26.016640 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016273 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf" containerName="extract" Jul 07 18:56:26.016640 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016283 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="65401510-654a-4e3c-b89b-809ee03e398c" containerName="extract" Jul 07 18:56:26.016640 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.016292 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="67ee7edd-7a6b-46a6-869e-d2054008e73c" containerName="extract" Jul 07 18:56:26.018540 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.018525 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:26.020906 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.020887 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4l2bz\"" Jul 07 18:56:26.057644 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.057618 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5"] Jul 07 18:56:26.159957 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.159931 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c234c716-5da6-40f9-87af-2ef2cee0cb5f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g4ft5\" (UID: \"c234c716-5da6-40f9-87af-2ef2cee0cb5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:26.160083 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.159985 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7wg2\" (UniqueName: \"kubernetes.io/projected/c234c716-5da6-40f9-87af-2ef2cee0cb5f-kube-api-access-d7wg2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g4ft5\" (UID: \"c234c716-5da6-40f9-87af-2ef2cee0cb5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:26.260278 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.260255 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7wg2\" (UniqueName: \"kubernetes.io/projected/c234c716-5da6-40f9-87af-2ef2cee0cb5f-kube-api-access-d7wg2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g4ft5\" (UID: \"c234c716-5da6-40f9-87af-2ef2cee0cb5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:26.260391 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.260310 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c234c716-5da6-40f9-87af-2ef2cee0cb5f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g4ft5\" (UID: \"c234c716-5da6-40f9-87af-2ef2cee0cb5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:26.260653 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.260637 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c234c716-5da6-40f9-87af-2ef2cee0cb5f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g4ft5\" (UID: \"c234c716-5da6-40f9-87af-2ef2cee0cb5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:26.269732 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.269682 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7wg2\" (UniqueName: \"kubernetes.io/projected/c234c716-5da6-40f9-87af-2ef2cee0cb5f-kube-api-access-d7wg2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g4ft5\" (UID: \"c234c716-5da6-40f9-87af-2ef2cee0cb5f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:26.328819 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.328802 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:26.675750 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:26.675726 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5"] Jul 07 18:56:26.678491 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:56:26.678461 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc234c716_5da6_40f9_87af_2ef2cee0cb5f.slice/crio-76934e3c80f23d4c0befb209c48f7d989af2d85994a5c8b9f579b576fd2ee713 WatchSource:0}: Error finding container 76934e3c80f23d4c0befb209c48f7d989af2d85994a5c8b9f579b576fd2ee713: Status 404 returned error can't find the container with id 76934e3c80f23d4c0befb209c48f7d989af2d85994a5c8b9f579b576fd2ee713 Jul 07 18:56:27.040644 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:27.040607 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" event={"ID":"c234c716-5da6-40f9-87af-2ef2cee0cb5f","Type":"ContainerStarted","Data":"76934e3c80f23d4c0befb209c48f7d989af2d85994a5c8b9f579b576fd2ee713"} Jul 07 18:56:28.026889 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:28.026856 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh"] Jul 07 18:56:28.029375 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:28.029358 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh" Jul 07 18:56:28.032768 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:28.032743 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-nqd49\"" Jul 07 18:56:28.047051 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:28.047027 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh"] Jul 07 18:56:28.174822 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:28.174791 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9wpm\" (UniqueName: \"kubernetes.io/projected/d8f98114-342f-4ee9-b918-c6796ae9e958-kube-api-access-t9wpm\") pod \"limitador-operator-controller-manager-859d7b78bd-l8njh\" (UID: \"d8f98114-342f-4ee9-b918-c6796ae9e958\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh" Jul 07 18:56:28.275929 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:28.275902 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9wpm\" (UniqueName: \"kubernetes.io/projected/d8f98114-342f-4ee9-b918-c6796ae9e958-kube-api-access-t9wpm\") pod \"limitador-operator-controller-manager-859d7b78bd-l8njh\" (UID: \"d8f98114-342f-4ee9-b918-c6796ae9e958\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh" Jul 07 18:56:28.288360 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:28.288302 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9wpm\" (UniqueName: \"kubernetes.io/projected/d8f98114-342f-4ee9-b918-c6796ae9e958-kube-api-access-t9wpm\") pod \"limitador-operator-controller-manager-859d7b78bd-l8njh\" (UID: \"d8f98114-342f-4ee9-b918-c6796ae9e958\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh" Jul 07 18:56:28.339721 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:28.339684 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh" Jul 07 18:56:28.737323 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:28.737163 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh"] Jul 07 18:56:28.740501 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:56:28.740459 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8f98114_342f_4ee9_b918_c6796ae9e958.slice/crio-462c8bb5dcb947c10addf5ad6e6219353f4972a2bcc336ca84d8ecc7674a8cb8 WatchSource:0}: Error finding container 462c8bb5dcb947c10addf5ad6e6219353f4972a2bcc336ca84d8ecc7674a8cb8: Status 404 returned error can't find the container with id 462c8bb5dcb947c10addf5ad6e6219353f4972a2bcc336ca84d8ecc7674a8cb8 Jul 07 18:56:29.053131 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:29.053097 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh" event={"ID":"d8f98114-342f-4ee9-b918-c6796ae9e958","Type":"ContainerStarted","Data":"462c8bb5dcb947c10addf5ad6e6219353f4972a2bcc336ca84d8ecc7674a8cb8"} Jul 07 18:56:30.525867 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:30.525840 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-pxmnc"] Jul 07 18:56:30.529070 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:30.529053 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pxmnc" Jul 07 18:56:30.531382 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:30.531363 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-hbkfp\"" Jul 07 18:56:30.546940 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:30.546921 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-pxmnc"] Jul 07 18:56:30.696454 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:30.696379 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh7qd\" (UniqueName: \"kubernetes.io/projected/e2c66c7f-f736-4cee-a590-9a3c64686947-kube-api-access-dh7qd\") pod \"authorino-operator-75cc8d5cb7-pxmnc\" (UID: \"e2c66c7f-f736-4cee-a590-9a3c64686947\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-pxmnc" Jul 07 18:56:30.797259 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:30.797227 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh7qd\" (UniqueName: \"kubernetes.io/projected/e2c66c7f-f736-4cee-a590-9a3c64686947-kube-api-access-dh7qd\") pod \"authorino-operator-75cc8d5cb7-pxmnc\" (UID: \"e2c66c7f-f736-4cee-a590-9a3c64686947\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-pxmnc" Jul 07 18:56:30.808884 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:30.808848 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh7qd\" (UniqueName: \"kubernetes.io/projected/e2c66c7f-f736-4cee-a590-9a3c64686947-kube-api-access-dh7qd\") pod \"authorino-operator-75cc8d5cb7-pxmnc\" (UID: \"e2c66c7f-f736-4cee-a590-9a3c64686947\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-pxmnc" Jul 07 18:56:30.838822 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:30.838798 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pxmnc" Jul 07 18:56:31.066432 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:31.066395 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh" event={"ID":"d8f98114-342f-4ee9-b918-c6796ae9e958","Type":"ContainerStarted","Data":"616ceb7fed088cc07c76c59e673f19372115388eaca000748f66af991fb43681"} Jul 07 18:56:31.066626 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:31.066479 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh" Jul 07 18:56:31.087044 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:31.087001 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh" podStartSLOduration=1.383231431 podStartE2EDuration="3.086986723s" totalImagesPullingTime="1.703755292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:56:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:56:31.084624576 +0000 UTC m=+398.503984821" watchObservedRunningTime="2026-07-07 18:56:31.086986723 +0000 UTC m=+398.506346948" Jul 07 18:56:31.191166 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:31.191133 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-pxmnc"] Jul 07 18:56:32.275503 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:56:32.275471 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2c66c7f_f736_4cee_a590_9a3c64686947.slice/crio-c9443a3709d072b159381d556c845db79d9951d31f38140a3eeb4ea7069c6029 WatchSource:0}: Error finding container c9443a3709d072b159381d556c845db79d9951d31f38140a3eeb4ea7069c6029: Status 404 returned error can't find the container with id c9443a3709d072b159381d556c845db79d9951d31f38140a3eeb4ea7069c6029 Jul 07 18:56:33.079772 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:33.079742 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" event={"ID":"c234c716-5da6-40f9-87af-2ef2cee0cb5f","Type":"ContainerStarted","Data":"d167adfa804ecd54f40548afe117417e0118c1c47d3aa03f15870c0146af7004"} Jul 07 18:56:33.079954 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:33.079880 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:33.081310 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:33.081284 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pxmnc" event={"ID":"e2c66c7f-f736-4cee-a590-9a3c64686947","Type":"ContainerStarted","Data":"c9443a3709d072b159381d556c845db79d9951d31f38140a3eeb4ea7069c6029"} Jul 07 18:56:33.106624 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:33.106549 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" podStartSLOduration=2.455101901 podStartE2EDuration="8.106536035s" totalImagesPullingTime="5.651434134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:56:33.103794207 +0000 UTC m=+400.523154433" watchObservedRunningTime="2026-07-07 18:56:33.106536035 +0000 UTC m=+400.525896260" Jul 07 18:56:33.917868 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:33.917824 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4wnh5"] Jul 07 18:56:33.928914 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:33.928848 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4wnh5" Jul 07 18:56:33.931340 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:33.931312 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5srgw\"" Jul 07 18:56:33.931474 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:33.931321 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 07 18:56:33.932269 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:33.932215 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4wnh5"] Jul 07 18:56:34.027077 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:34.027044 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xxw4\" (UniqueName: \"kubernetes.io/projected/46db6b91-a179-4f25-bd7b-7a00183e071c-kube-api-access-2xxw4\") pod \"dns-operator-controller-manager-864dd78c4c-4wnh5\" (UID: \"46db6b91-a179-4f25-bd7b-7a00183e071c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4wnh5" Jul 07 18:56:34.128144 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:34.128108 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xxw4\" (UniqueName: \"kubernetes.io/projected/46db6b91-a179-4f25-bd7b-7a00183e071c-kube-api-access-2xxw4\") pod \"dns-operator-controller-manager-864dd78c4c-4wnh5\" (UID: \"46db6b91-a179-4f25-bd7b-7a00183e071c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4wnh5" Jul 07 18:56:34.141128 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:34.141099 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xxw4\" (UniqueName: \"kubernetes.io/projected/46db6b91-a179-4f25-bd7b-7a00183e071c-kube-api-access-2xxw4\") pod \"dns-operator-controller-manager-864dd78c4c-4wnh5\" (UID: \"46db6b91-a179-4f25-bd7b-7a00183e071c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4wnh5" Jul 07 18:56:34.243191 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:34.243118 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4wnh5" Jul 07 18:56:34.645687 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:34.645661 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-4wnh5"] Jul 07 18:56:34.647598 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:56:34.647555 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46db6b91_a179_4f25_bd7b_7a00183e071c.slice/crio-f4369abc3117afd31fca861ec287b311b7591ebb68fb5febe8e460cf08d197aa WatchSource:0}: Error finding container f4369abc3117afd31fca861ec287b311b7591ebb68fb5febe8e460cf08d197aa: Status 404 returned error can't find the container with id f4369abc3117afd31fca861ec287b311b7591ebb68fb5febe8e460cf08d197aa Jul 07 18:56:35.094394 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:35.094359 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4wnh5" event={"ID":"46db6b91-a179-4f25-bd7b-7a00183e071c","Type":"ContainerStarted","Data":"f4369abc3117afd31fca861ec287b311b7591ebb68fb5febe8e460cf08d197aa"} Jul 07 18:56:35.095884 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:35.095853 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pxmnc" event={"ID":"e2c66c7f-f736-4cee-a590-9a3c64686947","Type":"ContainerStarted","Data":"640ecf02ebc0e7bdb93dc8b2f56fa693422d766f7c7578c192e05bc3dcaea804"} Jul 07 18:56:35.096016 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:35.095980 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pxmnc" Jul 07 18:56:35.118395 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:35.118347 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pxmnc" podStartSLOduration=3.041109264 podStartE2EDuration="5.1183339s" totalImagesPullingTime="2.077224636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:56:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:56:35.117380999 +0000 UTC m=+402.536741225" watchObservedRunningTime="2026-07-07 18:56:35.1183339 +0000 UTC m=+402.537694124" Jul 07 18:56:35.496223 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:35.496150 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-268dw"] Jul 07 18:56:35.499618 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:35.499596 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-268dw" Jul 07 18:56:35.502067 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:35.502014 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 07 18:56:35.502192 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:35.502127 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 07 18:56:35.502298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:35.502281 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-862l7\"" Jul 07 18:56:35.508174 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:35.508148 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-268dw"] Jul 07 18:56:35.644370 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:35.644284 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a19cc62-b466-4cd7-8cba-4f573f4c3035-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-268dw\" (UID: \"3a19cc62-b466-4cd7-8cba-4f573f4c3035\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-268dw" Jul 07 18:56:35.644370 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:35.644330 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28mkh\" (UniqueName: \"kubernetes.io/projected/3a19cc62-b466-4cd7-8cba-4f573f4c3035-kube-api-access-28mkh\") pod \"kuadrant-console-plugin-66b5774985-268dw\" (UID: \"3a19cc62-b466-4cd7-8cba-4f573f4c3035\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-268dw" Jul 07 18:56:35.644617 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:35.644387 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3a19cc62-b466-4cd7-8cba-4f573f4c3035-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-268dw\" (UID: \"3a19cc62-b466-4cd7-8cba-4f573f4c3035\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-268dw" Jul 07 18:56:35.745649 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:35.745619 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3a19cc62-b466-4cd7-8cba-4f573f4c3035-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-268dw\" (UID: \"3a19cc62-b466-4cd7-8cba-4f573f4c3035\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-268dw" Jul 07 18:56:35.745803 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:35.745774 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a19cc62-b466-4cd7-8cba-4f573f4c3035-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-268dw\" (UID: \"3a19cc62-b466-4cd7-8cba-4f573f4c3035\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-268dw" Jul 07 18:56:35.745803 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:35.745800 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28mkh\" (UniqueName: \"kubernetes.io/projected/3a19cc62-b466-4cd7-8cba-4f573f4c3035-kube-api-access-28mkh\") pod \"kuadrant-console-plugin-66b5774985-268dw\" (UID: \"3a19cc62-b466-4cd7-8cba-4f573f4c3035\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-268dw" Jul 07 18:56:35.745944 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:35.745924 2564 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 07 18:56:35.746018 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:35.746006 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a19cc62-b466-4cd7-8cba-4f573f4c3035-plugin-serving-cert podName:3a19cc62-b466-4cd7-8cba-4f573f4c3035 nodeName:}" failed. No retries permitted until 2026-07-07 18:56:36.245985121 +0000 UTC m=+403.665345325 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/3a19cc62-b466-4cd7-8cba-4f573f4c3035-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-268dw" (UID: "3a19cc62-b466-4cd7-8cba-4f573f4c3035") : secret "plugin-serving-cert" not found Jul 07 18:56:35.746248 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:35.746198 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3a19cc62-b466-4cd7-8cba-4f573f4c3035-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-268dw\" (UID: \"3a19cc62-b466-4cd7-8cba-4f573f4c3035\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-268dw" Jul 07 18:56:35.757462 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:35.757440 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28mkh\" (UniqueName: \"kubernetes.io/projected/3a19cc62-b466-4cd7-8cba-4f573f4c3035-kube-api-access-28mkh\") pod \"kuadrant-console-plugin-66b5774985-268dw\" (UID: \"3a19cc62-b466-4cd7-8cba-4f573f4c3035\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-268dw" Jul 07 18:56:36.249960 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.249926 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a19cc62-b466-4cd7-8cba-4f573f4c3035-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-268dw\" (UID: \"3a19cc62-b466-4cd7-8cba-4f573f4c3035\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-268dw" Jul 07 18:56:36.252893 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.252865 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a19cc62-b466-4cd7-8cba-4f573f4c3035-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-268dw\" (UID: \"3a19cc62-b466-4cd7-8cba-4f573f4c3035\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-268dw" Jul 07 18:56:36.268505 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.268476 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7975cb9647-jdmtd"] Jul 07 18:56:36.272614 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.272563 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.287285 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.287262 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7975cb9647-jdmtd"] Jul 07 18:56:36.412666 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.412644 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-268dw" Jul 07 18:56:36.452099 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.452066 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6aa77157-0983-4d33-9aa8-49fc8d996c5e-trusted-ca-bundle\") pod \"console-7975cb9647-jdmtd\" (UID: \"6aa77157-0983-4d33-9aa8-49fc8d996c5e\") " pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.452246 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.452117 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6aa77157-0983-4d33-9aa8-49fc8d996c5e-oauth-serving-cert\") pod \"console-7975cb9647-jdmtd\" (UID: \"6aa77157-0983-4d33-9aa8-49fc8d996c5e\") " pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.452246 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.452184 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz2gq\" (UniqueName: \"kubernetes.io/projected/6aa77157-0983-4d33-9aa8-49fc8d996c5e-kube-api-access-jz2gq\") pod \"console-7975cb9647-jdmtd\" (UID: \"6aa77157-0983-4d33-9aa8-49fc8d996c5e\") " pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.452369 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.452257 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6aa77157-0983-4d33-9aa8-49fc8d996c5e-console-serving-cert\") pod \"console-7975cb9647-jdmtd\" (UID: \"6aa77157-0983-4d33-9aa8-49fc8d996c5e\") " pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.452369 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.452292 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6aa77157-0983-4d33-9aa8-49fc8d996c5e-console-oauth-config\") pod \"console-7975cb9647-jdmtd\" (UID: \"6aa77157-0983-4d33-9aa8-49fc8d996c5e\") " pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.452369 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.452323 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6aa77157-0983-4d33-9aa8-49fc8d996c5e-console-config\") pod \"console-7975cb9647-jdmtd\" (UID: \"6aa77157-0983-4d33-9aa8-49fc8d996c5e\") " pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.452519 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.452391 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6aa77157-0983-4d33-9aa8-49fc8d996c5e-service-ca\") pod \"console-7975cb9647-jdmtd\" (UID: \"6aa77157-0983-4d33-9aa8-49fc8d996c5e\") " pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.553298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.553221 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6aa77157-0983-4d33-9aa8-49fc8d996c5e-trusted-ca-bundle\") pod \"console-7975cb9647-jdmtd\" (UID: \"6aa77157-0983-4d33-9aa8-49fc8d996c5e\") " pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.553298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.553266 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6aa77157-0983-4d33-9aa8-49fc8d996c5e-oauth-serving-cert\") pod \"console-7975cb9647-jdmtd\" (UID: \"6aa77157-0983-4d33-9aa8-49fc8d996c5e\") " pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.553298 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.553292 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz2gq\" (UniqueName: \"kubernetes.io/projected/6aa77157-0983-4d33-9aa8-49fc8d996c5e-kube-api-access-jz2gq\") pod \"console-7975cb9647-jdmtd\" (UID: \"6aa77157-0983-4d33-9aa8-49fc8d996c5e\") " pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.553549 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.553340 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6aa77157-0983-4d33-9aa8-49fc8d996c5e-console-serving-cert\") pod \"console-7975cb9647-jdmtd\" (UID: \"6aa77157-0983-4d33-9aa8-49fc8d996c5e\") " pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.553549 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.553380 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6aa77157-0983-4d33-9aa8-49fc8d996c5e-console-oauth-config\") pod \"console-7975cb9647-jdmtd\" (UID: \"6aa77157-0983-4d33-9aa8-49fc8d996c5e\") " pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.553549 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.553413 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6aa77157-0983-4d33-9aa8-49fc8d996c5e-console-config\") pod \"console-7975cb9647-jdmtd\" (UID: \"6aa77157-0983-4d33-9aa8-49fc8d996c5e\") " pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.553549 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.553476 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6aa77157-0983-4d33-9aa8-49fc8d996c5e-service-ca\") pod \"console-7975cb9647-jdmtd\" (UID: \"6aa77157-0983-4d33-9aa8-49fc8d996c5e\") " pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.554111 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.554052 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6aa77157-0983-4d33-9aa8-49fc8d996c5e-oauth-serving-cert\") pod \"console-7975cb9647-jdmtd\" (UID: \"6aa77157-0983-4d33-9aa8-49fc8d996c5e\") " pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.554239 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.554142 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6aa77157-0983-4d33-9aa8-49fc8d996c5e-service-ca\") pod \"console-7975cb9647-jdmtd\" (UID: \"6aa77157-0983-4d33-9aa8-49fc8d996c5e\") " pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.554239 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.554212 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6aa77157-0983-4d33-9aa8-49fc8d996c5e-trusted-ca-bundle\") pod \"console-7975cb9647-jdmtd\" (UID: \"6aa77157-0983-4d33-9aa8-49fc8d996c5e\") " pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.554341 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.554247 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6aa77157-0983-4d33-9aa8-49fc8d996c5e-console-config\") pod \"console-7975cb9647-jdmtd\" (UID: \"6aa77157-0983-4d33-9aa8-49fc8d996c5e\") " pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.556391 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.556374 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6aa77157-0983-4d33-9aa8-49fc8d996c5e-console-serving-cert\") pod \"console-7975cb9647-jdmtd\" (UID: \"6aa77157-0983-4d33-9aa8-49fc8d996c5e\") " pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.556529 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.556504 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6aa77157-0983-4d33-9aa8-49fc8d996c5e-console-oauth-config\") pod \"console-7975cb9647-jdmtd\" (UID: \"6aa77157-0983-4d33-9aa8-49fc8d996c5e\") " pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.563110 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.563087 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz2gq\" (UniqueName: \"kubernetes.io/projected/6aa77157-0983-4d33-9aa8-49fc8d996c5e-kube-api-access-jz2gq\") pod \"console-7975cb9647-jdmtd\" (UID: \"6aa77157-0983-4d33-9aa8-49fc8d996c5e\") " pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.584161 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.584142 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:36.762386 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.762360 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-268dw"] Jul 07 18:56:36.764311 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:56:36.764281 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a19cc62_b466_4cd7_8cba_4f573f4c3035.slice/crio-5fa1064d7dba6f3113ed5ce476c7515586f9a79c2c085cd86bdde11dfa363c5f WatchSource:0}: Error finding container 5fa1064d7dba6f3113ed5ce476c7515586f9a79c2c085cd86bdde11dfa363c5f: Status 404 returned error can't find the container with id 5fa1064d7dba6f3113ed5ce476c7515586f9a79c2c085cd86bdde11dfa363c5f Jul 07 18:56:36.936746 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:36.936718 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7975cb9647-jdmtd"] Jul 07 18:56:36.937894 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:56:36.937868 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aa77157_0983_4d33_9aa8_49fc8d996c5e.slice/crio-b6f34c76e672c730d7d141f2b04cd19007efb2944c1b6835549d11a98be2225c WatchSource:0}: Error finding container b6f34c76e672c730d7d141f2b04cd19007efb2944c1b6835549d11a98be2225c: Status 404 returned error can't find the container with id b6f34c76e672c730d7d141f2b04cd19007efb2944c1b6835549d11a98be2225c Jul 07 18:56:37.110156 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:37.110113 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4wnh5" event={"ID":"46db6b91-a179-4f25-bd7b-7a00183e071c","Type":"ContainerStarted","Data":"41351f3b6cbc021fae7657d6801f179e76093f29583b4503a41974b2fc097be0"} Jul 07 18:56:37.110350 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:37.110198 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4wnh5" Jul 07 18:56:37.111337 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:37.111313 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-268dw" event={"ID":"3a19cc62-b466-4cd7-8cba-4f573f4c3035","Type":"ContainerStarted","Data":"5fa1064d7dba6f3113ed5ce476c7515586f9a79c2c085cd86bdde11dfa363c5f"} Jul 07 18:56:37.112823 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:37.112803 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7975cb9647-jdmtd" event={"ID":"6aa77157-0983-4d33-9aa8-49fc8d996c5e","Type":"ContainerStarted","Data":"3add6252c0fc336dc0b61c9258283e7d6109361bf3b082ed00ce04b90dcae8f7"} Jul 07 18:56:37.112911 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:37.112836 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7975cb9647-jdmtd" event={"ID":"6aa77157-0983-4d33-9aa8-49fc8d996c5e","Type":"ContainerStarted","Data":"b6f34c76e672c730d7d141f2b04cd19007efb2944c1b6835549d11a98be2225c"} Jul 07 18:56:37.132330 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:37.132293 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4wnh5" podStartSLOduration=2.438716253 podStartE2EDuration="4.132281433s" totalImagesPullingTime="1.69356518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:56:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:56:37.130910419 +0000 UTC m=+404.550270644" watchObservedRunningTime="2026-07-07 18:56:37.132281433 +0000 UTC m=+404.551641657" Jul 07 18:56:37.153379 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:37.153337 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7975cb9647-jdmtd" podStartSLOduration=1.153324932 podStartE2EDuration="1.153324932s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:56:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:56:37.151247405 +0000 UTC m=+404.570607648" watchObservedRunningTime="2026-07-07 18:56:37.153324932 +0000 UTC m=+404.572685156" Jul 07 18:56:42.075200 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:42.075095 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh" Jul 07 18:56:44.090426 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:44.090392 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:45.928958 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:45.928924 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5"] Jul 07 18:56:45.929388 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:45.929136 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" podUID="c234c716-5da6-40f9-87af-2ef2cee0cb5f" containerName="manager" containerID="cri-o://d167adfa804ecd54f40548afe117417e0118c1c47d3aa03f15870c0146af7004" gracePeriod=2 Jul 07 18:56:45.940145 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:45.940115 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5"] Jul 07 18:56:45.955561 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:45.955532 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn"] Jul 07 18:56:45.956169 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:45.956121 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c234c716-5da6-40f9-87af-2ef2cee0cb5f" containerName="manager" Jul 07 18:56:45.956169 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:45.956142 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c234c716-5da6-40f9-87af-2ef2cee0cb5f" containerName="manager" Jul 07 18:56:45.956312 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:45.956245 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c234c716-5da6-40f9-87af-2ef2cee0cb5f" containerName="manager" Jul 07 18:56:45.959670 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:45.959647 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh"] Jul 07 18:56:45.959794 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:45.959767 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn" Jul 07 18:56:45.959863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:45.959837 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh" podUID="d8f98114-342f-4ee9-b918-c6796ae9e958" containerName="manager" containerID="cri-o://616ceb7fed088cc07c76c59e673f19372115388eaca000748f66af991fb43681" gracePeriod=2 Jul 07 18:56:45.964259 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:45.964236 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh"] Jul 07 18:56:45.974729 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:45.974707 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn"] Jul 07 18:56:45.981771 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:45.981750 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-swf96"] Jul 07 18:56:45.982268 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:45.982249 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8f98114-342f-4ee9-b918-c6796ae9e958" containerName="manager" Jul 07 18:56:45.982268 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:45.982268 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8f98114-342f-4ee9-b918-c6796ae9e958" containerName="manager" Jul 07 18:56:45.982415 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:45.982342 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8f98114-342f-4ee9-b918-c6796ae9e958" containerName="manager" Jul 07 18:56:45.987446 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:45.987420 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-swf96" Jul 07 18:56:45.997628 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:45.997605 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-swf96"] Jul 07 18:56:45.998759 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:45.998643 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-g4ft5\" is forbidden: User \"system:node:ip-10-0-142-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-66.ec2.internal' and this object" podUID="c234c716-5da6-40f9-87af-2ef2cee0cb5f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:46.000801 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:46.000769 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l8njh\" is forbidden: User \"system:node:ip-10-0-142-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-66.ec2.internal' and this object" podUID="d8f98114-342f-4ee9-b918-c6796ae9e958" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh" Jul 07 18:56:46.003272 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:46.003243 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l8njh\" is forbidden: User \"system:node:ip-10-0-142-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-66.ec2.internal' and this object" podUID="d8f98114-342f-4ee9-b918-c6796ae9e958" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh" Jul 07 18:56:46.025780 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:46.025742 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-g4ft5\" is forbidden: User \"system:node:ip-10-0-142-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-66.ec2.internal' and this object" podUID="c234c716-5da6-40f9-87af-2ef2cee0cb5f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:46.036245 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.036217 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2353973a-ee9b-42f4-9e6a-b31f89276f5a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r2dwn\" (UID: \"2353973a-ee9b-42f4-9e6a-b31f89276f5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn" Jul 07 18:56:46.036386 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.036363 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjr5n\" (UniqueName: \"kubernetes.io/projected/2353973a-ee9b-42f4-9e6a-b31f89276f5a-kube-api-access-gjr5n\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r2dwn\" (UID: \"2353973a-ee9b-42f4-9e6a-b31f89276f5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn" Jul 07 18:56:46.036470 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.036454 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgmjf\" (UniqueName: \"kubernetes.io/projected/12399cf6-ace4-452a-80e0-c74a24c16a41-kube-api-access-dgmjf\") pod \"limitador-operator-controller-manager-859d7b78bd-swf96\" (UID: \"12399cf6-ace4-452a-80e0-c74a24c16a41\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-swf96" Jul 07 18:56:46.105266 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.105235 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-pxmnc" Jul 07 18:56:46.134026 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:46.133990 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-g4ft5\" is forbidden: User \"system:node:ip-10-0-142-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-66.ec2.internal' and this object" podUID="c234c716-5da6-40f9-87af-2ef2cee0cb5f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:46.136667 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:46.136630 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l8njh\" is forbidden: User \"system:node:ip-10-0-142-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-66.ec2.internal' and this object" podUID="d8f98114-342f-4ee9-b918-c6796ae9e958" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh" Jul 07 18:56:46.137049 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.137024 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgmjf\" (UniqueName: \"kubernetes.io/projected/12399cf6-ace4-452a-80e0-c74a24c16a41-kube-api-access-dgmjf\") pod \"limitador-operator-controller-manager-859d7b78bd-swf96\" (UID: \"12399cf6-ace4-452a-80e0-c74a24c16a41\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-swf96" Jul 07 18:56:46.137145 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.137120 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2353973a-ee9b-42f4-9e6a-b31f89276f5a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r2dwn\" (UID: \"2353973a-ee9b-42f4-9e6a-b31f89276f5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn" Jul 07 18:56:46.137269 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.137199 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjr5n\" (UniqueName: \"kubernetes.io/projected/2353973a-ee9b-42f4-9e6a-b31f89276f5a-kube-api-access-gjr5n\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r2dwn\" (UID: \"2353973a-ee9b-42f4-9e6a-b31f89276f5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn" Jul 07 18:56:46.138102 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.138074 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2353973a-ee9b-42f4-9e6a-b31f89276f5a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r2dwn\" (UID: \"2353973a-ee9b-42f4-9e6a-b31f89276f5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn" Jul 07 18:56:46.154046 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.154001 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgmjf\" (UniqueName: \"kubernetes.io/projected/12399cf6-ace4-452a-80e0-c74a24c16a41-kube-api-access-dgmjf\") pod \"limitador-operator-controller-manager-859d7b78bd-swf96\" (UID: \"12399cf6-ace4-452a-80e0-c74a24c16a41\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-swf96" Jul 07 18:56:46.154425 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.154405 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjr5n\" (UniqueName: \"kubernetes.io/projected/2353973a-ee9b-42f4-9e6a-b31f89276f5a-kube-api-access-gjr5n\") pod \"kuadrant-operator-controller-manager-8684c9ff49-r2dwn\" (UID: \"2353973a-ee9b-42f4-9e6a-b31f89276f5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn" Jul 07 18:56:46.181205 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.181151 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn" Jul 07 18:56:46.188160 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.188113 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-swf96" Jul 07 18:56:46.195709 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.195689 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259"] Jul 07 18:56:46.200375 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.200357 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259" Jul 07 18:56:46.202443 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:46.202412 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-g4ft5\" is forbidden: User \"system:node:ip-10-0-142-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-66.ec2.internal' and this object" podUID="c234c716-5da6-40f9-87af-2ef2cee0cb5f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:46.213115 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.213084 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259"] Jul 07 18:56:46.226934 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:46.226899 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l8njh\" is forbidden: User \"system:node:ip-10-0-142-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-66.ec2.internal' and this object" podUID="d8f98114-342f-4ee9-b918-c6796ae9e958" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh" Jul 07 18:56:46.238518 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.238491 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfce2c8e-9a17-49b5-92a3-9469dabd0a2f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bq259\" (UID: \"dfce2c8e-9a17-49b5-92a3-9469dabd0a2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259" Jul 07 18:56:46.238648 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.238539 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbfvc\" (UniqueName: \"kubernetes.io/projected/dfce2c8e-9a17-49b5-92a3-9469dabd0a2f-kube-api-access-pbfvc\") pod \"kuadrant-operator-controller-manager-86878c769f-bq259\" (UID: \"dfce2c8e-9a17-49b5-92a3-9469dabd0a2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259" Jul 07 18:56:46.339785 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.339754 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfce2c8e-9a17-49b5-92a3-9469dabd0a2f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bq259\" (UID: \"dfce2c8e-9a17-49b5-92a3-9469dabd0a2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259" Jul 07 18:56:46.339955 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.339796 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbfvc\" (UniqueName: \"kubernetes.io/projected/dfce2c8e-9a17-49b5-92a3-9469dabd0a2f-kube-api-access-pbfvc\") pod \"kuadrant-operator-controller-manager-86878c769f-bq259\" (UID: \"dfce2c8e-9a17-49b5-92a3-9469dabd0a2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259" Jul 07 18:56:46.340159 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.340137 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfce2c8e-9a17-49b5-92a3-9469dabd0a2f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bq259\" (UID: \"dfce2c8e-9a17-49b5-92a3-9469dabd0a2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259" Jul 07 18:56:46.353849 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.353820 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbfvc\" (UniqueName: \"kubernetes.io/projected/dfce2c8e-9a17-49b5-92a3-9469dabd0a2f-kube-api-access-pbfvc\") pod \"kuadrant-operator-controller-manager-86878c769f-bq259\" (UID: \"dfce2c8e-9a17-49b5-92a3-9469dabd0a2f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259" Jul 07 18:56:46.512300 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.512260 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259" Jul 07 18:56:46.584336 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.584302 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:46.584544 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.584367 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:46.590101 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:46.590078 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:46.592166 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:46.592128 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-g4ft5\" is forbidden: User \"system:node:ip-10-0-142-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-66.ec2.internal' and this object" podUID="c234c716-5da6-40f9-87af-2ef2cee0cb5f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:46.594115 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:46.594088 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l8njh\" is forbidden: User \"system:node:ip-10-0-142-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-66.ec2.internal' and this object" podUID="d8f98114-342f-4ee9-b918-c6796ae9e958" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh" Jul 07 18:56:47.189998 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:47.189971 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7975cb9647-jdmtd" Jul 07 18:56:47.259050 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:47.258924 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67b458d9f9-hz952"] Jul 07 18:56:48.123158 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:48.123129 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-4wnh5" Jul 07 18:56:58.868649 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:58.868621 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:58.870915 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:58.870890 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-g4ft5\" is forbidden: User \"system:node:ip-10-0-142-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-66.ec2.internal' and this object" podUID="c234c716-5da6-40f9-87af-2ef2cee0cb5f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:58.912733 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:58.912708 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh" Jul 07 18:56:58.914919 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:58.914893 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l8njh\" is forbidden: User \"system:node:ip-10-0-142-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-66.ec2.internal' and this object" podUID="d8f98114-342f-4ee9-b918-c6796ae9e958" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh" Jul 07 18:56:58.916880 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:58.916848 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-g4ft5\" is forbidden: User \"system:node:ip-10-0-142-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-66.ec2.internal' and this object" podUID="c234c716-5da6-40f9-87af-2ef2cee0cb5f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:58.942144 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:58.942113 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn"] Jul 07 18:56:58.942791 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:56:58.942767 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2353973a_ee9b_42f4_9e6a_b31f89276f5a.slice/crio-c05e5ef4279d0caadba1cf09f5f17c70ae7e09bb25ad87ae7b95784ed6c065b3 WatchSource:0}: Error finding container c05e5ef4279d0caadba1cf09f5f17c70ae7e09bb25ad87ae7b95784ed6c065b3: Status 404 returned error can't find the container with id c05e5ef4279d0caadba1cf09f5f17c70ae7e09bb25ad87ae7b95784ed6c065b3 Jul 07 18:56:58.976684 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:58.976659 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c234c716-5da6-40f9-87af-2ef2cee0cb5f-extensions-socket-volume\") pod \"c234c716-5da6-40f9-87af-2ef2cee0cb5f\" (UID: \"c234c716-5da6-40f9-87af-2ef2cee0cb5f\") " Jul 07 18:56:58.976814 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:58.976707 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9wpm\" (UniqueName: \"kubernetes.io/projected/d8f98114-342f-4ee9-b918-c6796ae9e958-kube-api-access-t9wpm\") pod \"d8f98114-342f-4ee9-b918-c6796ae9e958\" (UID: \"d8f98114-342f-4ee9-b918-c6796ae9e958\") " Jul 07 18:56:58.976814 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:58.976795 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7wg2\" (UniqueName: \"kubernetes.io/projected/c234c716-5da6-40f9-87af-2ef2cee0cb5f-kube-api-access-d7wg2\") pod \"c234c716-5da6-40f9-87af-2ef2cee0cb5f\" (UID: \"c234c716-5da6-40f9-87af-2ef2cee0cb5f\") " Jul 07 18:56:58.977272 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:58.977240 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c234c716-5da6-40f9-87af-2ef2cee0cb5f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c234c716-5da6-40f9-87af-2ef2cee0cb5f" (UID: "c234c716-5da6-40f9-87af-2ef2cee0cb5f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:56:58.979256 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:58.979228 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c234c716-5da6-40f9-87af-2ef2cee0cb5f-kube-api-access-d7wg2" (OuterVolumeSpecName: "kube-api-access-d7wg2") pod "c234c716-5da6-40f9-87af-2ef2cee0cb5f" (UID: "c234c716-5da6-40f9-87af-2ef2cee0cb5f"). InnerVolumeSpecName "kube-api-access-d7wg2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:56:58.979336 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:58.979281 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8f98114-342f-4ee9-b918-c6796ae9e958-kube-api-access-t9wpm" (OuterVolumeSpecName: "kube-api-access-t9wpm") pod "d8f98114-342f-4ee9-b918-c6796ae9e958" (UID: "d8f98114-342f-4ee9-b918-c6796ae9e958"). InnerVolumeSpecName "kube-api-access-t9wpm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:56:59.025832 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.025811 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-swf96"] Jul 07 18:56:59.027183 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:56:59.027163 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12399cf6_ace4_452a_80e0_c74a24c16a41.slice/crio-f121d59139f07df24ac82c53608f97853fcdd01d7a17cf09a5dd63c20d7019c9 WatchSource:0}: Error finding container f121d59139f07df24ac82c53608f97853fcdd01d7a17cf09a5dd63c20d7019c9: Status 404 returned error can't find the container with id f121d59139f07df24ac82c53608f97853fcdd01d7a17cf09a5dd63c20d7019c9 Jul 07 18:56:59.033165 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.033148 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259"] Jul 07 18:56:59.035240 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:56:59.035218 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfce2c8e_9a17_49b5_92a3_9469dabd0a2f.slice/crio-c6e48f683191c42958e009ce24a74d00badc7794b4c3dcfd0db2a39b701407ec WatchSource:0}: Error finding container c6e48f683191c42958e009ce24a74d00badc7794b4c3dcfd0db2a39b701407ec: Status 404 returned error can't find the container with id c6e48f683191c42958e009ce24a74d00badc7794b4c3dcfd0db2a39b701407ec Jul 07 18:56:59.077707 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.077684 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7wg2\" (UniqueName: \"kubernetes.io/projected/c234c716-5da6-40f9-87af-2ef2cee0cb5f-kube-api-access-d7wg2\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:56:59.077707 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.077707 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c234c716-5da6-40f9-87af-2ef2cee0cb5f-extensions-socket-volume\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:56:59.077882 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.077716 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9wpm\" (UniqueName: \"kubernetes.io/projected/d8f98114-342f-4ee9-b918-c6796ae9e958-kube-api-access-t9wpm\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:56:59.104320 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.104292 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c234c716-5da6-40f9-87af-2ef2cee0cb5f" path="/var/lib/kubelet/pods/c234c716-5da6-40f9-87af-2ef2cee0cb5f/volumes" Jul 07 18:56:59.104643 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.104628 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8f98114-342f-4ee9-b918-c6796ae9e958" path="/var/lib/kubelet/pods/d8f98114-342f-4ee9-b918-c6796ae9e958/volumes" Jul 07 18:56:59.262024 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.261986 2564 generic.go:356] "Generic (PLEG): container finished" podID="c234c716-5da6-40f9-87af-2ef2cee0cb5f" containerID="d167adfa804ecd54f40548afe117417e0118c1c47d3aa03f15870c0146af7004" exitCode=0 Jul 07 18:56:59.262226 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.262037 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:59.262226 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.262048 2564 scope.go:117] "RemoveContainer" containerID="d167adfa804ecd54f40548afe117417e0118c1c47d3aa03f15870c0146af7004" Jul 07 18:56:59.264351 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:59.264321 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-g4ft5\" is forbidden: User \"system:node:ip-10-0-142-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-66.ec2.internal' and this object" podUID="c234c716-5da6-40f9-87af-2ef2cee0cb5f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:59.265887 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.265400 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-swf96" event={"ID":"12399cf6-ace4-452a-80e0-c74a24c16a41","Type":"ContainerStarted","Data":"98d5a50570fc052a07d13f16444f87badd0667aebddac05ec7f0491c14828f02"} Jul 07 18:56:59.265887 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.265449 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-swf96" event={"ID":"12399cf6-ace4-452a-80e0-c74a24c16a41","Type":"ContainerStarted","Data":"f121d59139f07df24ac82c53608f97853fcdd01d7a17cf09a5dd63c20d7019c9"} Jul 07 18:56:59.265887 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.265476 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-swf96" Jul 07 18:56:59.267274 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:59.267082 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-g4ft5\" is forbidden: User \"system:node:ip-10-0-142-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-66.ec2.internal' and this object" podUID="c234c716-5da6-40f9-87af-2ef2cee0cb5f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:59.268121 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.267637 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259" event={"ID":"dfce2c8e-9a17-49b5-92a3-9469dabd0a2f","Type":"ContainerStarted","Data":"6e7a3ee990b47f29b4e3e4ba596b2f85b1105415f76c80296f60668a7d0e12b8"} Jul 07 18:56:59.268121 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.267688 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259" event={"ID":"dfce2c8e-9a17-49b5-92a3-9469dabd0a2f","Type":"ContainerStarted","Data":"c6e48f683191c42958e009ce24a74d00badc7794b4c3dcfd0db2a39b701407ec"} Jul 07 18:56:59.268121 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.267902 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259" Jul 07 18:56:59.268937 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:59.268911 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-g4ft5\" is forbidden: User \"system:node:ip-10-0-142-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-66.ec2.internal' and this object" podUID="c234c716-5da6-40f9-87af-2ef2cee0cb5f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:59.270902 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.270878 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn" event={"ID":"2353973a-ee9b-42f4-9e6a-b31f89276f5a","Type":"ContainerStarted","Data":"a64300c93e2769da00d8dd706e5761a230b5c9c107fc8d419c2a50d5a242a103"} Jul 07 18:56:59.271002 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.270927 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn" event={"ID":"2353973a-ee9b-42f4-9e6a-b31f89276f5a","Type":"ContainerStarted","Data":"c05e5ef4279d0caadba1cf09f5f17c70ae7e09bb25ad87ae7b95784ed6c065b3"} Jul 07 18:56:59.271002 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.270959 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn" Jul 07 18:56:59.272762 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.272707 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-268dw" event={"ID":"3a19cc62-b466-4cd7-8cba-4f573f4c3035","Type":"ContainerStarted","Data":"7fa9b6520c678e16d797d77ad1f361ea23c19243d1212cb48b3c64e707c3775c"} Jul 07 18:56:59.274190 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.274168 2564 generic.go:356] "Generic (PLEG): container finished" podID="d8f98114-342f-4ee9-b918-c6796ae9e958" containerID="616ceb7fed088cc07c76c59e673f19372115388eaca000748f66af991fb43681" exitCode=0 Jul 07 18:56:59.274280 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.274205 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh" Jul 07 18:56:59.274462 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.274407 2564 scope.go:117] "RemoveContainer" containerID="d167adfa804ecd54f40548afe117417e0118c1c47d3aa03f15870c0146af7004" Jul 07 18:56:59.274755 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:59.274711 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d167adfa804ecd54f40548afe117417e0118c1c47d3aa03f15870c0146af7004\": container with ID starting with d167adfa804ecd54f40548afe117417e0118c1c47d3aa03f15870c0146af7004 not found: ID does not exist" containerID="d167adfa804ecd54f40548afe117417e0118c1c47d3aa03f15870c0146af7004" Jul 07 18:56:59.274755 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.274735 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d167adfa804ecd54f40548afe117417e0118c1c47d3aa03f15870c0146af7004"} err="failed to get container status \"d167adfa804ecd54f40548afe117417e0118c1c47d3aa03f15870c0146af7004\": rpc error: code = NotFound desc = could not find container \"d167adfa804ecd54f40548afe117417e0118c1c47d3aa03f15870c0146af7004\": container with ID starting with d167adfa804ecd54f40548afe117417e0118c1c47d3aa03f15870c0146af7004 not found: ID does not exist" Jul 07 18:56:59.274755 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.274755 2564 scope.go:117] "RemoveContainer" containerID="616ceb7fed088cc07c76c59e673f19372115388eaca000748f66af991fb43681" Jul 07 18:56:59.284509 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.284488 2564 scope.go:117] "RemoveContainer" containerID="616ceb7fed088cc07c76c59e673f19372115388eaca000748f66af991fb43681" Jul 07 18:56:59.284791 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:59.284764 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"616ceb7fed088cc07c76c59e673f19372115388eaca000748f66af991fb43681\": container with ID starting with 616ceb7fed088cc07c76c59e673f19372115388eaca000748f66af991fb43681 not found: ID does not exist" containerID="616ceb7fed088cc07c76c59e673f19372115388eaca000748f66af991fb43681" Jul 07 18:56:59.284872 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.284800 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"616ceb7fed088cc07c76c59e673f19372115388eaca000748f66af991fb43681"} err="failed to get container status \"616ceb7fed088cc07c76c59e673f19372115388eaca000748f66af991fb43681\": rpc error: code = NotFound desc = could not find container \"616ceb7fed088cc07c76c59e673f19372115388eaca000748f66af991fb43681\": container with ID starting with 616ceb7fed088cc07c76c59e673f19372115388eaca000748f66af991fb43681 not found: ID does not exist" Jul 07 18:56:59.289780 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:59.289756 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-l8njh\" is forbidden: User \"system:node:ip-10-0-142-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-66.ec2.internal' and this object" podUID="d8f98114-342f-4ee9-b918-c6796ae9e958" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-l8njh" Jul 07 18:56:59.290354 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.290318 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-swf96" podStartSLOduration=14.290304791 podStartE2EDuration="14.290304791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:56:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:56:59.287762152 +0000 UTC m=+426.707122377" watchObservedRunningTime="2026-07-07 18:56:59.290304791 +0000 UTC m=+426.709664993" Jul 07 18:56:59.313122 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.313086 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn" podStartSLOduration=14.31307446 podStartE2EDuration="14.31307446s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:56:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:56:59.311328278 +0000 UTC m=+426.730688515" watchObservedRunningTime="2026-07-07 18:56:59.31307446 +0000 UTC m=+426.732434681" Jul 07 18:56:59.326820 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.326783 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-268dw" podStartSLOduration=2.507425263 podStartE2EDuration="24.326770913s" totalImagesPullingTime="21.81934565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:56:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:56:59.32628296 +0000 UTC m=+426.745643186" watchObservedRunningTime="2026-07-07 18:56:59.326770913 +0000 UTC m=+426.746131137" Jul 07 18:56:59.328084 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:56:59.328063 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-g4ft5\" is forbidden: User \"system:node:ip-10-0-142-66.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-66.ec2.internal' and this object" podUID="c234c716-5da6-40f9-87af-2ef2cee0cb5f" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g4ft5" Jul 07 18:56:59.353639 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:56:59.353595 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259" podStartSLOduration=13.353564375 podStartE2EDuration="13.353564375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:56:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:56:59.352203493 +0000 UTC m=+426.771563721" watchObservedRunningTime="2026-07-07 18:56:59.353564375 +0000 UTC m=+426.772924665" Jul 07 18:57:10.284708 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:10.284676 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-swf96" Jul 07 18:57:10.285417 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:10.285397 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn" Jul 07 18:57:10.285617 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:10.285603 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259" Jul 07 18:57:10.384070 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:10.384044 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn"] Jul 07 18:57:10.384253 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:10.384229 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn" podUID="2353973a-ee9b-42f4-9e6a-b31f89276f5a" containerName="manager" containerID="cri-o://a64300c93e2769da00d8dd706e5761a230b5c9c107fc8d419c2a50d5a242a103" gracePeriod=10 Jul 07 18:57:10.802466 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:10.802412 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn" Jul 07 18:57:10.882406 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:10.882372 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2353973a-ee9b-42f4-9e6a-b31f89276f5a-extensions-socket-volume\") pod \"2353973a-ee9b-42f4-9e6a-b31f89276f5a\" (UID: \"2353973a-ee9b-42f4-9e6a-b31f89276f5a\") " Jul 07 18:57:10.882562 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:10.882478 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjr5n\" (UniqueName: \"kubernetes.io/projected/2353973a-ee9b-42f4-9e6a-b31f89276f5a-kube-api-access-gjr5n\") pod \"2353973a-ee9b-42f4-9e6a-b31f89276f5a\" (UID: \"2353973a-ee9b-42f4-9e6a-b31f89276f5a\") " Jul 07 18:57:10.882765 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:10.882740 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2353973a-ee9b-42f4-9e6a-b31f89276f5a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2353973a-ee9b-42f4-9e6a-b31f89276f5a" (UID: "2353973a-ee9b-42f4-9e6a-b31f89276f5a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:57:10.884557 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:10.884539 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2353973a-ee9b-42f4-9e6a-b31f89276f5a-kube-api-access-gjr5n" (OuterVolumeSpecName: "kube-api-access-gjr5n") pod "2353973a-ee9b-42f4-9e6a-b31f89276f5a" (UID: "2353973a-ee9b-42f4-9e6a-b31f89276f5a"). InnerVolumeSpecName "kube-api-access-gjr5n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:57:10.983972 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:10.983949 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjr5n\" (UniqueName: \"kubernetes.io/projected/2353973a-ee9b-42f4-9e6a-b31f89276f5a-kube-api-access-gjr5n\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:57:10.983972 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:10.983970 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2353973a-ee9b-42f4-9e6a-b31f89276f5a-extensions-socket-volume\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:57:11.342890 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:11.342859 2564 generic.go:356] "Generic (PLEG): container finished" podID="2353973a-ee9b-42f4-9e6a-b31f89276f5a" containerID="a64300c93e2769da00d8dd706e5761a230b5c9c107fc8d419c2a50d5a242a103" exitCode=0 Jul 07 18:57:11.343267 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:11.342901 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn" Jul 07 18:57:11.343267 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:11.342946 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn" event={"ID":"2353973a-ee9b-42f4-9e6a-b31f89276f5a","Type":"ContainerDied","Data":"a64300c93e2769da00d8dd706e5761a230b5c9c107fc8d419c2a50d5a242a103"} Jul 07 18:57:11.343267 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:11.342999 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn" event={"ID":"2353973a-ee9b-42f4-9e6a-b31f89276f5a","Type":"ContainerDied","Data":"c05e5ef4279d0caadba1cf09f5f17c70ae7e09bb25ad87ae7b95784ed6c065b3"} Jul 07 18:57:11.343267 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:11.343028 2564 scope.go:117] "RemoveContainer" containerID="a64300c93e2769da00d8dd706e5761a230b5c9c107fc8d419c2a50d5a242a103" Jul 07 18:57:11.351900 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:11.351881 2564 scope.go:117] "RemoveContainer" containerID="a64300c93e2769da00d8dd706e5761a230b5c9c107fc8d419c2a50d5a242a103" Jul 07 18:57:11.352141 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:57:11.352123 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a64300c93e2769da00d8dd706e5761a230b5c9c107fc8d419c2a50d5a242a103\": container with ID starting with a64300c93e2769da00d8dd706e5761a230b5c9c107fc8d419c2a50d5a242a103 not found: ID does not exist" containerID="a64300c93e2769da00d8dd706e5761a230b5c9c107fc8d419c2a50d5a242a103" Jul 07 18:57:11.352187 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:11.352149 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a64300c93e2769da00d8dd706e5761a230b5c9c107fc8d419c2a50d5a242a103"} err="failed to get container status \"a64300c93e2769da00d8dd706e5761a230b5c9c107fc8d419c2a50d5a242a103\": rpc error: code = NotFound desc = could not find container \"a64300c93e2769da00d8dd706e5761a230b5c9c107fc8d419c2a50d5a242a103\": container with ID starting with a64300c93e2769da00d8dd706e5761a230b5c9c107fc8d419c2a50d5a242a103 not found: ID does not exist" Jul 07 18:57:11.366084 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:11.366038 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn"] Jul 07 18:57:11.376917 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:11.376896 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-r2dwn"] Jul 07 18:57:12.289464 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:12.289431 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-67b458d9f9-hz952" podUID="badcd60d-49a4-454d-85ea-0a28672e4379" containerName="console" containerID="cri-o://9a56c84438df7a6f9c039a27aabf38a4d1fbef3d9d80aa13c57fd8f1a4fea044" gracePeriod=15 Jul 07 18:57:12.734100 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:12.734072 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:57:12.899478 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:12.899450 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-oauth-serving-cert\") pod \"badcd60d-49a4-454d-85ea-0a28672e4379\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " Jul 07 18:57:12.899682 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:12.899490 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/badcd60d-49a4-454d-85ea-0a28672e4379-console-serving-cert\") pod \"badcd60d-49a4-454d-85ea-0a28672e4379\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " Jul 07 18:57:12.899682 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:12.899515 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-console-config\") pod \"badcd60d-49a4-454d-85ea-0a28672e4379\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " Jul 07 18:57:12.899682 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:12.899533 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-service-ca\") pod \"badcd60d-49a4-454d-85ea-0a28672e4379\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " Jul 07 18:57:12.899682 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:12.899640 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctxxs\" (UniqueName: \"kubernetes.io/projected/badcd60d-49a4-454d-85ea-0a28672e4379-kube-api-access-ctxxs\") pod \"badcd60d-49a4-454d-85ea-0a28672e4379\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " Jul 07 18:57:12.899682 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:12.899683 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/badcd60d-49a4-454d-85ea-0a28672e4379-console-oauth-config\") pod \"badcd60d-49a4-454d-85ea-0a28672e4379\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " Jul 07 18:57:12.899994 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:12.899756 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-trusted-ca-bundle\") pod \"badcd60d-49a4-454d-85ea-0a28672e4379\" (UID: \"badcd60d-49a4-454d-85ea-0a28672e4379\") " Jul 07 18:57:12.899994 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:12.899968 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-service-ca" (OuterVolumeSpecName: "service-ca") pod "badcd60d-49a4-454d-85ea-0a28672e4379" (UID: "badcd60d-49a4-454d-85ea-0a28672e4379"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:57:12.899994 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:12.899974 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-console-config" (OuterVolumeSpecName: "console-config") pod "badcd60d-49a4-454d-85ea-0a28672e4379" (UID: "badcd60d-49a4-454d-85ea-0a28672e4379"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:57:12.900153 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:12.899991 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "badcd60d-49a4-454d-85ea-0a28672e4379" (UID: "badcd60d-49a4-454d-85ea-0a28672e4379"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:57:12.900153 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:12.900098 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-oauth-serving-cert\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:57:12.900153 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:12.900114 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-console-config\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:57:12.900309 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:12.900127 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-service-ca\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:57:12.900402 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:12.900381 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "badcd60d-49a4-454d-85ea-0a28672e4379" (UID: "badcd60d-49a4-454d-85ea-0a28672e4379"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:57:12.901770 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:12.901748 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/badcd60d-49a4-454d-85ea-0a28672e4379-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "badcd60d-49a4-454d-85ea-0a28672e4379" (UID: "badcd60d-49a4-454d-85ea-0a28672e4379"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:57:12.901881 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:12.901863 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/badcd60d-49a4-454d-85ea-0a28672e4379-kube-api-access-ctxxs" (OuterVolumeSpecName: "kube-api-access-ctxxs") pod "badcd60d-49a4-454d-85ea-0a28672e4379" (UID: "badcd60d-49a4-454d-85ea-0a28672e4379"). InnerVolumeSpecName "kube-api-access-ctxxs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:57:12.901974 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:12.901954 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/badcd60d-49a4-454d-85ea-0a28672e4379-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "badcd60d-49a4-454d-85ea-0a28672e4379" (UID: "badcd60d-49a4-454d-85ea-0a28672e4379"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:57:13.001221 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:13.001197 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctxxs\" (UniqueName: \"kubernetes.io/projected/badcd60d-49a4-454d-85ea-0a28672e4379-kube-api-access-ctxxs\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:57:13.001221 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:13.001221 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/badcd60d-49a4-454d-85ea-0a28672e4379-console-oauth-config\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:57:13.001391 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:13.001230 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/badcd60d-49a4-454d-85ea-0a28672e4379-trusted-ca-bundle\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:57:13.001391 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:13.001240 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/badcd60d-49a4-454d-85ea-0a28672e4379-console-serving-cert\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:57:13.105932 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:13.105903 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2353973a-ee9b-42f4-9e6a-b31f89276f5a" path="/var/lib/kubelet/pods/2353973a-ee9b-42f4-9e6a-b31f89276f5a/volumes" Jul 07 18:57:13.355226 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:13.355145 2564 generic.go:356] "Generic (PLEG): container finished" podID="badcd60d-49a4-454d-85ea-0a28672e4379" containerID="9a56c84438df7a6f9c039a27aabf38a4d1fbef3d9d80aa13c57fd8f1a4fea044" exitCode=0 Jul 07 18:57:13.355226 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:13.355186 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67b458d9f9-hz952" Jul 07 18:57:13.355801 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:13.355229 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67b458d9f9-hz952" event={"ID":"badcd60d-49a4-454d-85ea-0a28672e4379","Type":"ContainerDied","Data":"9a56c84438df7a6f9c039a27aabf38a4d1fbef3d9d80aa13c57fd8f1a4fea044"} Jul 07 18:57:13.355801 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:13.355278 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67b458d9f9-hz952" event={"ID":"badcd60d-49a4-454d-85ea-0a28672e4379","Type":"ContainerDied","Data":"ecd51bba92d554e326a6f438c1aec0f808bc7cbfb8e5c3d08ef9f07f7855c6ce"} Jul 07 18:57:13.355801 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:13.355302 2564 scope.go:117] "RemoveContainer" containerID="9a56c84438df7a6f9c039a27aabf38a4d1fbef3d9d80aa13c57fd8f1a4fea044" Jul 07 18:57:13.364308 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:13.364287 2564 scope.go:117] "RemoveContainer" containerID="9a56c84438df7a6f9c039a27aabf38a4d1fbef3d9d80aa13c57fd8f1a4fea044" Jul 07 18:57:13.364605 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:57:13.364551 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a56c84438df7a6f9c039a27aabf38a4d1fbef3d9d80aa13c57fd8f1a4fea044\": container with ID starting with 9a56c84438df7a6f9c039a27aabf38a4d1fbef3d9d80aa13c57fd8f1a4fea044 not found: ID does not exist" containerID="9a56c84438df7a6f9c039a27aabf38a4d1fbef3d9d80aa13c57fd8f1a4fea044" Jul 07 18:57:13.364708 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:13.364611 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a56c84438df7a6f9c039a27aabf38a4d1fbef3d9d80aa13c57fd8f1a4fea044"} err="failed to get container status \"9a56c84438df7a6f9c039a27aabf38a4d1fbef3d9d80aa13c57fd8f1a4fea044\": rpc error: code = NotFound desc = could not find container \"9a56c84438df7a6f9c039a27aabf38a4d1fbef3d9d80aa13c57fd8f1a4fea044\": container with ID starting with 9a56c84438df7a6f9c039a27aabf38a4d1fbef3d9d80aa13c57fd8f1a4fea044 not found: ID does not exist" Jul 07 18:57:13.375331 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:13.375311 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67b458d9f9-hz952"] Jul 07 18:57:13.382008 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:13.381983 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67b458d9f9-hz952"] Jul 07 18:57:15.104503 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:15.104469 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="badcd60d-49a4-454d-85ea-0a28672e4379" path="/var/lib/kubelet/pods/badcd60d-49a4-454d-85ea-0a28672e4379/volumes" Jul 07 18:57:28.763418 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:28.763376 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-r2tws"] Jul 07 18:57:28.763906 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:28.763791 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2353973a-ee9b-42f4-9e6a-b31f89276f5a" containerName="manager" Jul 07 18:57:28.763906 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:28.763803 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2353973a-ee9b-42f4-9e6a-b31f89276f5a" containerName="manager" Jul 07 18:57:28.763906 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:28.763819 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="badcd60d-49a4-454d-85ea-0a28672e4379" containerName="console" Jul 07 18:57:28.763906 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:28.763825 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="badcd60d-49a4-454d-85ea-0a28672e4379" containerName="console" Jul 07 18:57:28.763906 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:28.763884 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2353973a-ee9b-42f4-9e6a-b31f89276f5a" containerName="manager" Jul 07 18:57:28.763906 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:28.763893 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="badcd60d-49a4-454d-85ea-0a28672e4379" containerName="console" Jul 07 18:57:28.775972 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:28.775949 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-r2tws"] Jul 07 18:57:28.776114 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:28.776034 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-r2tws" Jul 07 18:57:28.778426 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:28.778398 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 18:57:28.843541 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:28.843510 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ce747ceb-d682-4e98-a2a9-30b4cb11a49d-config-file\") pod \"limitador-limitador-64c94fd64f-r2tws\" (UID: \"ce747ceb-d682-4e98-a2a9-30b4cb11a49d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-r2tws" Jul 07 18:57:28.843713 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:28.843568 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfnh4\" (UniqueName: \"kubernetes.io/projected/ce747ceb-d682-4e98-a2a9-30b4cb11a49d-kube-api-access-qfnh4\") pod \"limitador-limitador-64c94fd64f-r2tws\" (UID: \"ce747ceb-d682-4e98-a2a9-30b4cb11a49d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-r2tws" Jul 07 18:57:28.857075 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:28.856714 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-r2tws"] Jul 07 18:57:28.944482 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:28.944451 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ce747ceb-d682-4e98-a2a9-30b4cb11a49d-config-file\") pod \"limitador-limitador-64c94fd64f-r2tws\" (UID: \"ce747ceb-d682-4e98-a2a9-30b4cb11a49d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-r2tws" Jul 07 18:57:28.944650 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:28.944510 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfnh4\" (UniqueName: \"kubernetes.io/projected/ce747ceb-d682-4e98-a2a9-30b4cb11a49d-kube-api-access-qfnh4\") pod \"limitador-limitador-64c94fd64f-r2tws\" (UID: \"ce747ceb-d682-4e98-a2a9-30b4cb11a49d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-r2tws" Jul 07 18:57:28.945178 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:28.945157 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ce747ceb-d682-4e98-a2a9-30b4cb11a49d-config-file\") pod \"limitador-limitador-64c94fd64f-r2tws\" (UID: \"ce747ceb-d682-4e98-a2a9-30b4cb11a49d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-r2tws" Jul 07 18:57:28.958416 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:28.958389 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfnh4\" (UniqueName: \"kubernetes.io/projected/ce747ceb-d682-4e98-a2a9-30b4cb11a49d-kube-api-access-qfnh4\") pod \"limitador-limitador-64c94fd64f-r2tws\" (UID: \"ce747ceb-d682-4e98-a2a9-30b4cb11a49d\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-r2tws" Jul 07 18:57:29.087763 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:29.087691 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-r2tws" Jul 07 18:57:29.464017 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:29.463949 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-r2tws"] Jul 07 18:57:29.467279 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:57:29.467255 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce747ceb_d682_4e98_a2a9_30b4cb11a49d.slice/crio-4666b26219a81cac56727ddaf27655bca24d3bc9141f56b4f4840e46c941be0d WatchSource:0}: Error finding container 4666b26219a81cac56727ddaf27655bca24d3bc9141f56b4f4840e46c941be0d: Status 404 returned error can't find the container with id 4666b26219a81cac56727ddaf27655bca24d3bc9141f56b4f4840e46c941be0d Jul 07 18:57:30.459119 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:30.459078 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-r2tws" event={"ID":"ce747ceb-d682-4e98-a2a9-30b4cb11a49d","Type":"ContainerStarted","Data":"4666b26219a81cac56727ddaf27655bca24d3bc9141f56b4f4840e46c941be0d"} Jul 07 18:57:33.487038 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:33.487001 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-r2tws" event={"ID":"ce747ceb-d682-4e98-a2a9-30b4cb11a49d","Type":"ContainerStarted","Data":"74ee1f8796c98597d56d6237dde9471cb038aac7b408ea14f2de4a040caf25d8"} Jul 07 18:57:33.487481 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:33.487070 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-r2tws" Jul 07 18:57:33.506372 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:33.506327 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-r2tws" podStartSLOduration=2.406200174 podStartE2EDuration="5.50630902s" totalImagesPullingTime="3.100108846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:57:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:57:33.503543894 +0000 UTC m=+460.922904138" watchObservedRunningTime="2026-07-07 18:57:33.50630902 +0000 UTC m=+460.925669430" Jul 07 18:57:43.730867 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:43.730838 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-r2tws"] Jul 07 18:57:43.731364 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:43.730997 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-r2tws" podUID="ce747ceb-d682-4e98-a2a9-30b4cb11a49d" containerName="limitador" containerID="cri-o://74ee1f8796c98597d56d6237dde9471cb038aac7b408ea14f2de4a040caf25d8" gracePeriod=30 Jul 07 18:57:43.731730 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:43.731688 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-r2tws" Jul 07 18:57:44.413223 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:44.413200 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-r2tws" Jul 07 18:57:44.474025 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:44.474000 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfnh4\" (UniqueName: \"kubernetes.io/projected/ce747ceb-d682-4e98-a2a9-30b4cb11a49d-kube-api-access-qfnh4\") pod \"ce747ceb-d682-4e98-a2a9-30b4cb11a49d\" (UID: \"ce747ceb-d682-4e98-a2a9-30b4cb11a49d\") " Jul 07 18:57:44.474181 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:44.474056 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ce747ceb-d682-4e98-a2a9-30b4cb11a49d-config-file\") pod \"ce747ceb-d682-4e98-a2a9-30b4cb11a49d\" (UID: \"ce747ceb-d682-4e98-a2a9-30b4cb11a49d\") " Jul 07 18:57:44.474459 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:44.474434 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce747ceb-d682-4e98-a2a9-30b4cb11a49d-config-file" (OuterVolumeSpecName: "config-file") pod "ce747ceb-d682-4e98-a2a9-30b4cb11a49d" (UID: "ce747ceb-d682-4e98-a2a9-30b4cb11a49d"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:57:44.476641 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:44.476611 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce747ceb-d682-4e98-a2a9-30b4cb11a49d-kube-api-access-qfnh4" (OuterVolumeSpecName: "kube-api-access-qfnh4") pod "ce747ceb-d682-4e98-a2a9-30b4cb11a49d" (UID: "ce747ceb-d682-4e98-a2a9-30b4cb11a49d"). InnerVolumeSpecName "kube-api-access-qfnh4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:57:44.547053 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:44.547029 2564 generic.go:356] "Generic (PLEG): container finished" podID="ce747ceb-d682-4e98-a2a9-30b4cb11a49d" containerID="74ee1f8796c98597d56d6237dde9471cb038aac7b408ea14f2de4a040caf25d8" exitCode=0 Jul 07 18:57:44.547142 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:44.547074 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-r2tws" Jul 07 18:57:44.547142 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:44.547101 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-r2tws" event={"ID":"ce747ceb-d682-4e98-a2a9-30b4cb11a49d","Type":"ContainerDied","Data":"74ee1f8796c98597d56d6237dde9471cb038aac7b408ea14f2de4a040caf25d8"} Jul 07 18:57:44.547256 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:44.547158 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-r2tws" event={"ID":"ce747ceb-d682-4e98-a2a9-30b4cb11a49d","Type":"ContainerDied","Data":"4666b26219a81cac56727ddaf27655bca24d3bc9141f56b4f4840e46c941be0d"} Jul 07 18:57:44.547256 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:44.547182 2564 scope.go:117] "RemoveContainer" containerID="74ee1f8796c98597d56d6237dde9471cb038aac7b408ea14f2de4a040caf25d8" Jul 07 18:57:44.559427 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:44.559409 2564 scope.go:117] "RemoveContainer" containerID="74ee1f8796c98597d56d6237dde9471cb038aac7b408ea14f2de4a040caf25d8" Jul 07 18:57:44.559721 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:57:44.559702 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74ee1f8796c98597d56d6237dde9471cb038aac7b408ea14f2de4a040caf25d8\": container with ID starting with 74ee1f8796c98597d56d6237dde9471cb038aac7b408ea14f2de4a040caf25d8 not found: ID does not exist" containerID="74ee1f8796c98597d56d6237dde9471cb038aac7b408ea14f2de4a040caf25d8" Jul 07 18:57:44.559789 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:44.559730 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74ee1f8796c98597d56d6237dde9471cb038aac7b408ea14f2de4a040caf25d8"} err="failed to get container status \"74ee1f8796c98597d56d6237dde9471cb038aac7b408ea14f2de4a040caf25d8\": rpc error: code = NotFound desc = could not find container \"74ee1f8796c98597d56d6237dde9471cb038aac7b408ea14f2de4a040caf25d8\": container with ID starting with 74ee1f8796c98597d56d6237dde9471cb038aac7b408ea14f2de4a040caf25d8 not found: ID does not exist" Jul 07 18:57:44.569780 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:44.569759 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-r2tws"] Jul 07 18:57:44.573958 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:44.573938 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-r2tws"] Jul 07 18:57:44.574901 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:44.574884 2564 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ce747ceb-d682-4e98-a2a9-30b4cb11a49d-config-file\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:57:44.574958 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:44.574906 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfnh4\" (UniqueName: \"kubernetes.io/projected/ce747ceb-d682-4e98-a2a9-30b4cb11a49d-kube-api-access-qfnh4\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:57:45.103678 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:45.103652 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce747ceb-d682-4e98-a2a9-30b4cb11a49d" path="/var/lib/kubelet/pods/ce747ceb-d682-4e98-a2a9-30b4cb11a49d/volumes" Jul 07 18:57:50.455915 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:50.455881 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-649zj"] Jul 07 18:57:50.456474 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:50.456244 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce747ceb-d682-4e98-a2a9-30b4cb11a49d" containerName="limitador" Jul 07 18:57:50.456474 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:50.456255 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce747ceb-d682-4e98-a2a9-30b4cb11a49d" containerName="limitador" Jul 07 18:57:50.456474 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:50.456322 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce747ceb-d682-4e98-a2a9-30b4cb11a49d" containerName="limitador" Jul 07 18:57:50.459610 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:50.459589 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-649zj" Jul 07 18:57:50.462050 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:50.462033 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 07 18:57:50.462181 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:50.462073 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7rx5h\"" Jul 07 18:57:50.473102 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:50.473076 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-649zj"] Jul 07 18:57:50.518730 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:50.518698 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvfd4\" (UniqueName: \"kubernetes.io/projected/122d33f0-a664-4055-9ded-4d2f5dcc4331-kube-api-access-qvfd4\") pod \"postgres-69fb4594cf-649zj\" (UID: \"122d33f0-a664-4055-9ded-4d2f5dcc4331\") " pod="opendatahub/postgres-69fb4594cf-649zj" Jul 07 18:57:50.518893 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:50.518747 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/122d33f0-a664-4055-9ded-4d2f5dcc4331-data\") pod \"postgres-69fb4594cf-649zj\" (UID: \"122d33f0-a664-4055-9ded-4d2f5dcc4331\") " pod="opendatahub/postgres-69fb4594cf-649zj" Jul 07 18:57:50.619910 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:50.619878 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/122d33f0-a664-4055-9ded-4d2f5dcc4331-data\") pod \"postgres-69fb4594cf-649zj\" (UID: \"122d33f0-a664-4055-9ded-4d2f5dcc4331\") " pod="opendatahub/postgres-69fb4594cf-649zj" Jul 07 18:57:50.620110 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:50.619964 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvfd4\" (UniqueName: \"kubernetes.io/projected/122d33f0-a664-4055-9ded-4d2f5dcc4331-kube-api-access-qvfd4\") pod \"postgres-69fb4594cf-649zj\" (UID: \"122d33f0-a664-4055-9ded-4d2f5dcc4331\") " pod="opendatahub/postgres-69fb4594cf-649zj" Jul 07 18:57:50.620313 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:50.620289 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/122d33f0-a664-4055-9ded-4d2f5dcc4331-data\") pod \"postgres-69fb4594cf-649zj\" (UID: \"122d33f0-a664-4055-9ded-4d2f5dcc4331\") " pod="opendatahub/postgres-69fb4594cf-649zj" Jul 07 18:57:50.631033 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:50.630998 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvfd4\" (UniqueName: \"kubernetes.io/projected/122d33f0-a664-4055-9ded-4d2f5dcc4331-kube-api-access-qvfd4\") pod \"postgres-69fb4594cf-649zj\" (UID: \"122d33f0-a664-4055-9ded-4d2f5dcc4331\") " pod="opendatahub/postgres-69fb4594cf-649zj" Jul 07 18:57:50.770269 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:50.770241 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-649zj" Jul 07 18:57:51.114031 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:51.114004 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-649zj"] Jul 07 18:57:51.115594 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:57:51.115547 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod122d33f0_a664_4055_9ded_4d2f5dcc4331.slice/crio-210822188d7e47e2e5c505986aaf0eb051983e5faac702a32793c702971ca628 WatchSource:0}: Error finding container 210822188d7e47e2e5c505986aaf0eb051983e5faac702a32793c702971ca628: Status 404 returned error can't find the container with id 210822188d7e47e2e5c505986aaf0eb051983e5faac702a32793c702971ca628 Jul 07 18:57:51.587755 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:51.587723 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-649zj" event={"ID":"122d33f0-a664-4055-9ded-4d2f5dcc4331","Type":"ContainerStarted","Data":"210822188d7e47e2e5c505986aaf0eb051983e5faac702a32793c702971ca628"} Jul 07 18:57:56.091242 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:56.091217 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 07 18:57:56.622960 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:56.622929 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-649zj" event={"ID":"122d33f0-a664-4055-9ded-4d2f5dcc4331","Type":"ContainerStarted","Data":"0455b27b2d1776886389feb32c9ca97fdace4a44d3417e3db254c304656e9505"} Jul 07 18:57:56.623139 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:56.623059 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-649zj" Jul 07 18:57:56.641005 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:57:56.640955 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-649zj" podStartSLOduration=1.6718072579999999 podStartE2EDuration="6.64094069s" totalImagesPullingTime="4.969133432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:57:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:57:56.63905361 +0000 UTC m=+484.058413832" watchObservedRunningTime="2026-07-07 18:57:56.64094069 +0000 UTC m=+484.060300916" Jul 07 18:58:02.659870 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:02.659838 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-649zj" Jul 07 18:58:09.441299 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.441263 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-fbfb9f759-vf4dl"] Jul 07 18:58:09.450558 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.450528 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-fbfb9f759-vf4dl" Jul 07 18:58:09.459608 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.459568 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 18:58:09.459723 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.459569 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-w7tt9\"" Jul 07 18:58:09.460307 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.460285 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 18:58:09.463955 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.463931 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-fbfb9f759-vf4dl"] Jul 07 18:58:09.594174 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.594135 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb3ce9dc-b3dd-4145-940e-4330ca21bcbf-webhook-cert\") pod \"maas-controller-fbfb9f759-vf4dl\" (UID: \"fb3ce9dc-b3dd-4145-940e-4330ca21bcbf\") " pod="opendatahub/maas-controller-fbfb9f759-vf4dl" Jul 07 18:58:09.594352 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.594199 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2jn4\" (UniqueName: \"kubernetes.io/projected/fb3ce9dc-b3dd-4145-940e-4330ca21bcbf-kube-api-access-d2jn4\") pod \"maas-controller-fbfb9f759-vf4dl\" (UID: \"fb3ce9dc-b3dd-4145-940e-4330ca21bcbf\") " pod="opendatahub/maas-controller-fbfb9f759-vf4dl" Jul 07 18:58:09.670693 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.670664 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d8f857c75-6xhvw"] Jul 07 18:58:09.674474 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.674451 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d8f857c75-6xhvw" Jul 07 18:58:09.692371 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.692309 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d8f857c75-6xhvw"] Jul 07 18:58:09.695043 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.695013 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2jn4\" (UniqueName: \"kubernetes.io/projected/fb3ce9dc-b3dd-4145-940e-4330ca21bcbf-kube-api-access-d2jn4\") pod \"maas-controller-fbfb9f759-vf4dl\" (UID: \"fb3ce9dc-b3dd-4145-940e-4330ca21bcbf\") " pod="opendatahub/maas-controller-fbfb9f759-vf4dl" Jul 07 18:58:09.695170 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.695120 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb3ce9dc-b3dd-4145-940e-4330ca21bcbf-webhook-cert\") pod \"maas-controller-fbfb9f759-vf4dl\" (UID: \"fb3ce9dc-b3dd-4145-940e-4330ca21bcbf\") " pod="opendatahub/maas-controller-fbfb9f759-vf4dl" Jul 07 18:58:09.697812 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.697791 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb3ce9dc-b3dd-4145-940e-4330ca21bcbf-webhook-cert\") pod \"maas-controller-fbfb9f759-vf4dl\" (UID: \"fb3ce9dc-b3dd-4145-940e-4330ca21bcbf\") " pod="opendatahub/maas-controller-fbfb9f759-vf4dl" Jul 07 18:58:09.711923 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.711898 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2jn4\" (UniqueName: \"kubernetes.io/projected/fb3ce9dc-b3dd-4145-940e-4330ca21bcbf-kube-api-access-d2jn4\") pod \"maas-controller-fbfb9f759-vf4dl\" (UID: \"fb3ce9dc-b3dd-4145-940e-4330ca21bcbf\") " pod="opendatahub/maas-controller-fbfb9f759-vf4dl" Jul 07 18:58:09.761449 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.761423 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-fbfb9f759-vf4dl" Jul 07 18:58:09.796040 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.796006 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ad58a61-cbb9-4497-994b-e7009ed93a80-webhook-cert\") pod \"maas-controller-d8f857c75-6xhvw\" (UID: \"2ad58a61-cbb9-4497-994b-e7009ed93a80\") " pod="opendatahub/maas-controller-d8f857c75-6xhvw" Jul 07 18:58:09.796040 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.796042 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqjk5\" (UniqueName: \"kubernetes.io/projected/2ad58a61-cbb9-4497-994b-e7009ed93a80-kube-api-access-wqjk5\") pod \"maas-controller-d8f857c75-6xhvw\" (UID: \"2ad58a61-cbb9-4497-994b-e7009ed93a80\") " pod="opendatahub/maas-controller-d8f857c75-6xhvw" Jul 07 18:58:09.897122 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.897088 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ad58a61-cbb9-4497-994b-e7009ed93a80-webhook-cert\") pod \"maas-controller-d8f857c75-6xhvw\" (UID: \"2ad58a61-cbb9-4497-994b-e7009ed93a80\") " pod="opendatahub/maas-controller-d8f857c75-6xhvw" Jul 07 18:58:09.897270 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.897129 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqjk5\" (UniqueName: \"kubernetes.io/projected/2ad58a61-cbb9-4497-994b-e7009ed93a80-kube-api-access-wqjk5\") pod \"maas-controller-d8f857c75-6xhvw\" (UID: \"2ad58a61-cbb9-4497-994b-e7009ed93a80\") " pod="opendatahub/maas-controller-d8f857c75-6xhvw" Jul 07 18:58:09.899663 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.899633 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ad58a61-cbb9-4497-994b-e7009ed93a80-webhook-cert\") pod \"maas-controller-d8f857c75-6xhvw\" (UID: \"2ad58a61-cbb9-4497-994b-e7009ed93a80\") " pod="opendatahub/maas-controller-d8f857c75-6xhvw" Jul 07 18:58:09.906251 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.906226 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqjk5\" (UniqueName: \"kubernetes.io/projected/2ad58a61-cbb9-4497-994b-e7009ed93a80-kube-api-access-wqjk5\") pod \"maas-controller-d8f857c75-6xhvw\" (UID: \"2ad58a61-cbb9-4497-994b-e7009ed93a80\") " pod="opendatahub/maas-controller-d8f857c75-6xhvw" Jul 07 18:58:09.984311 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:09.984232 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d8f857c75-6xhvw" Jul 07 18:58:10.113046 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:10.113019 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-fbfb9f759-vf4dl"] Jul 07 18:58:10.115434 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:58:10.115397 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb3ce9dc_b3dd_4145_940e_4330ca21bcbf.slice/crio-ab55786fba31eaa51f511f79656a7e57f4b7334207e984002454b2e4b2f28ba7 WatchSource:0}: Error finding container ab55786fba31eaa51f511f79656a7e57f4b7334207e984002454b2e4b2f28ba7: Status 404 returned error can't find the container with id ab55786fba31eaa51f511f79656a7e57f4b7334207e984002454b2e4b2f28ba7 Jul 07 18:58:10.334229 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:10.334201 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d8f857c75-6xhvw"] Jul 07 18:58:10.335464 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:58:10.335439 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ad58a61_cbb9_4497_994b_e7009ed93a80.slice/crio-f970e050c17773a52a7697586b483096aaab43d36c3d718f6e4f65a4d5e8c709 WatchSource:0}: Error finding container f970e050c17773a52a7697586b483096aaab43d36c3d718f6e4f65a4d5e8c709: Status 404 returned error can't find the container with id f970e050c17773a52a7697586b483096aaab43d36c3d718f6e4f65a4d5e8c709 Jul 07 18:58:10.708910 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:10.708817 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-fbfb9f759-vf4dl" event={"ID":"fb3ce9dc-b3dd-4145-940e-4330ca21bcbf","Type":"ContainerStarted","Data":"ab55786fba31eaa51f511f79656a7e57f4b7334207e984002454b2e4b2f28ba7"} Jul 07 18:58:10.711791 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:10.711755 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d8f857c75-6xhvw" event={"ID":"2ad58a61-cbb9-4497-994b-e7009ed93a80","Type":"ContainerStarted","Data":"f970e050c17773a52a7697586b483096aaab43d36c3d718f6e4f65a4d5e8c709"} Jul 07 18:58:12.728868 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:12.728760 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d8f857c75-6xhvw" event={"ID":"2ad58a61-cbb9-4497-994b-e7009ed93a80","Type":"ContainerStarted","Data":"b754d13e9a4189cb9497afbf78130650ce6de1ed41c842196abe61519309b5a7"} Jul 07 18:58:12.728868 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:12.728832 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d8f857c75-6xhvw" Jul 07 18:58:12.730355 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:12.730330 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-fbfb9f759-vf4dl" event={"ID":"fb3ce9dc-b3dd-4145-940e-4330ca21bcbf","Type":"ContainerStarted","Data":"2ef1d4273ebb7be3bb57b61fd282c83c946fc16d4e76364047194c0c175ad41b"} Jul 07 18:58:12.730482 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:12.730454 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-fbfb9f759-vf4dl" Jul 07 18:58:12.777322 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:12.777278 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d8f857c75-6xhvw" podStartSLOduration=1.738787544 podStartE2EDuration="3.777263189s" totalImagesPullingTime="2.038475645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:58:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:58:12.774643026 +0000 UTC m=+500.194003244" watchObservedRunningTime="2026-07-07 18:58:12.777263189 +0000 UTC m=+500.196623411" Jul 07 18:58:23.743036 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:23.743007 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d8f857c75-6xhvw" Jul 07 18:58:23.743490 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:23.743086 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-fbfb9f759-vf4dl" Jul 07 18:58:23.763191 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:23.763131 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-fbfb9f759-vf4dl" podStartSLOduration=12.515986055 podStartE2EDuration="14.763111613s" totalImagesPullingTime="2.247125558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:58:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:58:12.814997525 +0000 UTC m=+500.234357750" watchObservedRunningTime="2026-07-07 18:58:23.763111613 +0000 UTC m=+511.182471838" Jul 07 18:58:23.809919 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:23.809887 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-fbfb9f759-vf4dl"] Jul 07 18:58:23.810201 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:23.810155 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-fbfb9f759-vf4dl" podUID="fb3ce9dc-b3dd-4145-940e-4330ca21bcbf" containerName="manager" containerID="cri-o://2ef1d4273ebb7be3bb57b61fd282c83c946fc16d4e76364047194c0c175ad41b" gracePeriod=30 Jul 07 18:58:24.210209 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:24.210181 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-fbfb9f759-vf4dl" Jul 07 18:58:24.322213 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:24.322182 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb3ce9dc-b3dd-4145-940e-4330ca21bcbf-webhook-cert\") pod \"fb3ce9dc-b3dd-4145-940e-4330ca21bcbf\" (UID: \"fb3ce9dc-b3dd-4145-940e-4330ca21bcbf\") " Jul 07 18:58:24.322378 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:24.322278 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2jn4\" (UniqueName: \"kubernetes.io/projected/fb3ce9dc-b3dd-4145-940e-4330ca21bcbf-kube-api-access-d2jn4\") pod \"fb3ce9dc-b3dd-4145-940e-4330ca21bcbf\" (UID: \"fb3ce9dc-b3dd-4145-940e-4330ca21bcbf\") " Jul 07 18:58:24.324537 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:24.324507 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb3ce9dc-b3dd-4145-940e-4330ca21bcbf-kube-api-access-d2jn4" (OuterVolumeSpecName: "kube-api-access-d2jn4") pod "fb3ce9dc-b3dd-4145-940e-4330ca21bcbf" (UID: "fb3ce9dc-b3dd-4145-940e-4330ca21bcbf"). InnerVolumeSpecName "kube-api-access-d2jn4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:58:24.324537 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:24.324509 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb3ce9dc-b3dd-4145-940e-4330ca21bcbf-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "fb3ce9dc-b3dd-4145-940e-4330ca21bcbf" (UID: "fb3ce9dc-b3dd-4145-940e-4330ca21bcbf"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:58:24.423212 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:24.423188 2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fb3ce9dc-b3dd-4145-940e-4330ca21bcbf-webhook-cert\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:58:24.423212 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:24.423212 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2jn4\" (UniqueName: \"kubernetes.io/projected/fb3ce9dc-b3dd-4145-940e-4330ca21bcbf-kube-api-access-d2jn4\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 18:58:24.800701 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:24.800672 2564 generic.go:356] "Generic (PLEG): container finished" podID="fb3ce9dc-b3dd-4145-940e-4330ca21bcbf" containerID="2ef1d4273ebb7be3bb57b61fd282c83c946fc16d4e76364047194c0c175ad41b" exitCode=0 Jul 07 18:58:24.801068 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:24.800712 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-fbfb9f759-vf4dl" Jul 07 18:58:24.801068 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:24.800754 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-fbfb9f759-vf4dl" event={"ID":"fb3ce9dc-b3dd-4145-940e-4330ca21bcbf","Type":"ContainerDied","Data":"2ef1d4273ebb7be3bb57b61fd282c83c946fc16d4e76364047194c0c175ad41b"} Jul 07 18:58:24.801068 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:24.800828 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-fbfb9f759-vf4dl" event={"ID":"fb3ce9dc-b3dd-4145-940e-4330ca21bcbf","Type":"ContainerDied","Data":"ab55786fba31eaa51f511f79656a7e57f4b7334207e984002454b2e4b2f28ba7"} Jul 07 18:58:24.801068 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:24.800851 2564 scope.go:117] "RemoveContainer" containerID="2ef1d4273ebb7be3bb57b61fd282c83c946fc16d4e76364047194c0c175ad41b" Jul 07 18:58:24.810806 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:24.810637 2564 scope.go:117] "RemoveContainer" containerID="2ef1d4273ebb7be3bb57b61fd282c83c946fc16d4e76364047194c0c175ad41b" Jul 07 18:58:24.810905 ip-10-0-142-66 kubenswrapper[2564]: E0707 18:58:24.810891 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ef1d4273ebb7be3bb57b61fd282c83c946fc16d4e76364047194c0c175ad41b\": container with ID starting with 2ef1d4273ebb7be3bb57b61fd282c83c946fc16d4e76364047194c0c175ad41b not found: ID does not exist" containerID="2ef1d4273ebb7be3bb57b61fd282c83c946fc16d4e76364047194c0c175ad41b" Jul 07 18:58:24.810943 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:24.810914 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ef1d4273ebb7be3bb57b61fd282c83c946fc16d4e76364047194c0c175ad41b"} err="failed to get container status \"2ef1d4273ebb7be3bb57b61fd282c83c946fc16d4e76364047194c0c175ad41b\": rpc error: code = NotFound desc = could not find container \"2ef1d4273ebb7be3bb57b61fd282c83c946fc16d4e76364047194c0c175ad41b\": container with ID starting with 2ef1d4273ebb7be3bb57b61fd282c83c946fc16d4e76364047194c0c175ad41b not found: ID does not exist" Jul 07 18:58:24.822651 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:24.822630 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-fbfb9f759-vf4dl"] Jul 07 18:58:24.826811 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:24.826792 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-fbfb9f759-vf4dl"] Jul 07 18:58:25.103195 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:25.103121 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb3ce9dc-b3dd-4145-940e-4330ca21bcbf" path="/var/lib/kubelet/pods/fb3ce9dc-b3dd-4145-940e-4330ca21bcbf/volumes" Jul 07 18:58:44.626779 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:44.626691 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-6d85fdd848-krbqg"] Jul 07 18:58:44.627276 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:44.627256 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb3ce9dc-b3dd-4145-940e-4330ca21bcbf" containerName="manager" Jul 07 18:58:44.627363 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:44.627280 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb3ce9dc-b3dd-4145-940e-4330ca21bcbf" containerName="manager" Jul 07 18:58:44.627416 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:44.627392 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb3ce9dc-b3dd-4145-940e-4330ca21bcbf" containerName="manager" Jul 07 18:58:44.645846 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:44.645812 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6d85fdd848-krbqg"] Jul 07 18:58:44.645975 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:44.645917 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6d85fdd848-krbqg" Jul 07 18:58:44.648891 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:44.648867 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 07 18:58:44.793637 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:44.793606 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/218f5aaa-287e-4243-ba90-a62a1912b472-maas-api-tls\") pod \"maas-api-6d85fdd848-krbqg\" (UID: \"218f5aaa-287e-4243-ba90-a62a1912b472\") " pod="opendatahub/maas-api-6d85fdd848-krbqg" Jul 07 18:58:44.793637 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:44.793640 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6sv2\" (UniqueName: \"kubernetes.io/projected/218f5aaa-287e-4243-ba90-a62a1912b472-kube-api-access-h6sv2\") pod \"maas-api-6d85fdd848-krbqg\" (UID: \"218f5aaa-287e-4243-ba90-a62a1912b472\") " pod="opendatahub/maas-api-6d85fdd848-krbqg" Jul 07 18:58:44.793874 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:44.793774 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/218f5aaa-287e-4243-ba90-a62a1912b472-openshift-service-ca\") pod \"maas-api-6d85fdd848-krbqg\" (UID: \"218f5aaa-287e-4243-ba90-a62a1912b472\") " pod="opendatahub/maas-api-6d85fdd848-krbqg" Jul 07 18:58:44.895188 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:44.895076 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/218f5aaa-287e-4243-ba90-a62a1912b472-maas-api-tls\") pod \"maas-api-6d85fdd848-krbqg\" (UID: \"218f5aaa-287e-4243-ba90-a62a1912b472\") " pod="opendatahub/maas-api-6d85fdd848-krbqg" Jul 07 18:58:44.895188 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:44.895117 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6sv2\" (UniqueName: \"kubernetes.io/projected/218f5aaa-287e-4243-ba90-a62a1912b472-kube-api-access-h6sv2\") pod \"maas-api-6d85fdd848-krbqg\" (UID: \"218f5aaa-287e-4243-ba90-a62a1912b472\") " pod="opendatahub/maas-api-6d85fdd848-krbqg" Jul 07 18:58:44.895448 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:44.895204 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/218f5aaa-287e-4243-ba90-a62a1912b472-openshift-service-ca\") pod \"maas-api-6d85fdd848-krbqg\" (UID: \"218f5aaa-287e-4243-ba90-a62a1912b472\") " pod="opendatahub/maas-api-6d85fdd848-krbqg" Jul 07 18:58:44.896071 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:44.896042 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/218f5aaa-287e-4243-ba90-a62a1912b472-openshift-service-ca\") pod \"maas-api-6d85fdd848-krbqg\" (UID: \"218f5aaa-287e-4243-ba90-a62a1912b472\") " pod="opendatahub/maas-api-6d85fdd848-krbqg" Jul 07 18:58:44.897812 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:44.897791 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/218f5aaa-287e-4243-ba90-a62a1912b472-maas-api-tls\") pod \"maas-api-6d85fdd848-krbqg\" (UID: \"218f5aaa-287e-4243-ba90-a62a1912b472\") " pod="opendatahub/maas-api-6d85fdd848-krbqg" Jul 07 18:58:44.903187 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:44.903165 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6sv2\" (UniqueName: \"kubernetes.io/projected/218f5aaa-287e-4243-ba90-a62a1912b472-kube-api-access-h6sv2\") pod \"maas-api-6d85fdd848-krbqg\" (UID: \"218f5aaa-287e-4243-ba90-a62a1912b472\") " pod="opendatahub/maas-api-6d85fdd848-krbqg" Jul 07 18:58:44.958310 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:44.958290 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6d85fdd848-krbqg" Jul 07 18:58:45.153903 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.153828 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-474mg"] Jul 07 18:58:45.156567 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.156547 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-474mg" Jul 07 18:58:45.158876 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.158854 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 07 18:58:45.168882 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.168855 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-474mg"] Jul 07 18:58:45.180669 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.180615 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-wzdjn"] Jul 07 18:58:45.185244 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.185224 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-wzdjn" Jul 07 18:58:45.186489 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.186449 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-wzdjn"] Jul 07 18:58:45.187707 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.187687 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-m7mgd\"" Jul 07 18:58:45.299059 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.299020 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13d7fd32-6207-4178-ac83-914df4da7e9f-plugins-config-volume\") pod \"payload-processing-58f86c7798-wzdjn\" (UID: \"13d7fd32-6207-4178-ac83-914df4da7e9f\") " pod="openshift-ingress/payload-processing-58f86c7798-wzdjn" Jul 07 18:58:45.299201 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.299073 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdm26\" (UniqueName: \"kubernetes.io/projected/13d7fd32-6207-4178-ac83-914df4da7e9f-kube-api-access-wdm26\") pod \"payload-processing-58f86c7798-wzdjn\" (UID: \"13d7fd32-6207-4178-ac83-914df4da7e9f\") " pod="openshift-ingress/payload-processing-58f86c7798-wzdjn" Jul 07 18:58:45.299201 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.299144 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0fba180-a064-4905-86b3-df268d3d51c8-plugins-config-volume\") pod \"payload-pre-processing-7964457696-474mg\" (UID: \"c0fba180-a064-4905-86b3-df268d3d51c8\") " pod="openshift-ingress/payload-pre-processing-7964457696-474mg" Jul 07 18:58:45.299268 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.299223 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbf4m\" (UniqueName: \"kubernetes.io/projected/c0fba180-a064-4905-86b3-df268d3d51c8-kube-api-access-wbf4m\") pod \"payload-pre-processing-7964457696-474mg\" (UID: \"c0fba180-a064-4905-86b3-df268d3d51c8\") " pod="openshift-ingress/payload-pre-processing-7964457696-474mg" Jul 07 18:58:45.325028 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.325007 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6d85fdd848-krbqg"] Jul 07 18:58:45.326916 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:58:45.326885 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod218f5aaa_287e_4243_ba90_a62a1912b472.slice/crio-4a74e912f8812114ddad5000cb002d3112521929756ed74c82af9ecf89320c6f WatchSource:0}: Error finding container 4a74e912f8812114ddad5000cb002d3112521929756ed74c82af9ecf89320c6f: Status 404 returned error can't find the container with id 4a74e912f8812114ddad5000cb002d3112521929756ed74c82af9ecf89320c6f Jul 07 18:58:45.399833 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.399799 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13d7fd32-6207-4178-ac83-914df4da7e9f-plugins-config-volume\") pod \"payload-processing-58f86c7798-wzdjn\" (UID: \"13d7fd32-6207-4178-ac83-914df4da7e9f\") " pod="openshift-ingress/payload-processing-58f86c7798-wzdjn" Jul 07 18:58:45.399977 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.399838 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdm26\" (UniqueName: \"kubernetes.io/projected/13d7fd32-6207-4178-ac83-914df4da7e9f-kube-api-access-wdm26\") pod \"payload-processing-58f86c7798-wzdjn\" (UID: \"13d7fd32-6207-4178-ac83-914df4da7e9f\") " pod="openshift-ingress/payload-processing-58f86c7798-wzdjn" Jul 07 18:58:45.399977 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.399869 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0fba180-a064-4905-86b3-df268d3d51c8-plugins-config-volume\") pod \"payload-pre-processing-7964457696-474mg\" (UID: \"c0fba180-a064-4905-86b3-df268d3d51c8\") " pod="openshift-ingress/payload-pre-processing-7964457696-474mg" Jul 07 18:58:45.399977 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.399910 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbf4m\" (UniqueName: \"kubernetes.io/projected/c0fba180-a064-4905-86b3-df268d3d51c8-kube-api-access-wbf4m\") pod \"payload-pre-processing-7964457696-474mg\" (UID: \"c0fba180-a064-4905-86b3-df268d3d51c8\") " pod="openshift-ingress/payload-pre-processing-7964457696-474mg" Jul 07 18:58:45.400652 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.400631 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13d7fd32-6207-4178-ac83-914df4da7e9f-plugins-config-volume\") pod \"payload-processing-58f86c7798-wzdjn\" (UID: \"13d7fd32-6207-4178-ac83-914df4da7e9f\") " pod="openshift-ingress/payload-processing-58f86c7798-wzdjn" Jul 07 18:58:45.400755 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.400736 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0fba180-a064-4905-86b3-df268d3d51c8-plugins-config-volume\") pod \"payload-pre-processing-7964457696-474mg\" (UID: \"c0fba180-a064-4905-86b3-df268d3d51c8\") " pod="openshift-ingress/payload-pre-processing-7964457696-474mg" Jul 07 18:58:45.408545 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.408493 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdm26\" (UniqueName: \"kubernetes.io/projected/13d7fd32-6207-4178-ac83-914df4da7e9f-kube-api-access-wdm26\") pod \"payload-processing-58f86c7798-wzdjn\" (UID: \"13d7fd32-6207-4178-ac83-914df4da7e9f\") " pod="openshift-ingress/payload-processing-58f86c7798-wzdjn" Jul 07 18:58:45.408545 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.408506 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbf4m\" (UniqueName: \"kubernetes.io/projected/c0fba180-a064-4905-86b3-df268d3d51c8-kube-api-access-wbf4m\") pod \"payload-pre-processing-7964457696-474mg\" (UID: \"c0fba180-a064-4905-86b3-df268d3d51c8\") " pod="openshift-ingress/payload-pre-processing-7964457696-474mg" Jul 07 18:58:45.468366 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.468339 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-474mg" Jul 07 18:58:45.496312 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.496286 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-wzdjn" Jul 07 18:58:45.829177 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.829152 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-474mg"] Jul 07 18:58:45.830939 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:58:45.830915 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0fba180_a064_4905_86b3_df268d3d51c8.slice/crio-7df64bcb04b6d54375164aa36c3ddb3f168ae806f863f10fc833acb343844094 WatchSource:0}: Error finding container 7df64bcb04b6d54375164aa36c3ddb3f168ae806f863f10fc833acb343844094: Status 404 returned error can't find the container with id 7df64bcb04b6d54375164aa36c3ddb3f168ae806f863f10fc833acb343844094 Jul 07 18:58:45.886817 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.886792 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-wzdjn"] Jul 07 18:58:45.887940 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:58:45.887917 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13d7fd32_6207_4178_ac83_914df4da7e9f.slice/crio-64e6d121830649d640ad2e0033c45696aaae935e333a1a1a0566b665c35b15e8 WatchSource:0}: Error finding container 64e6d121830649d640ad2e0033c45696aaae935e333a1a1a0566b665c35b15e8: Status 404 returned error can't find the container with id 64e6d121830649d640ad2e0033c45696aaae935e333a1a1a0566b665c35b15e8 Jul 07 18:58:45.921666 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.921637 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-wzdjn" event={"ID":"13d7fd32-6207-4178-ac83-914df4da7e9f","Type":"ContainerStarted","Data":"64e6d121830649d640ad2e0033c45696aaae935e333a1a1a0566b665c35b15e8"} Jul 07 18:58:45.922842 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.922819 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6d85fdd848-krbqg" event={"ID":"218f5aaa-287e-4243-ba90-a62a1912b472","Type":"ContainerStarted","Data":"4a74e912f8812114ddad5000cb002d3112521929756ed74c82af9ecf89320c6f"} Jul 07 18:58:45.923947 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:45.923922 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-474mg" event={"ID":"c0fba180-a064-4905-86b3-df268d3d51c8","Type":"ContainerStarted","Data":"7df64bcb04b6d54375164aa36c3ddb3f168ae806f863f10fc833acb343844094"} Jul 07 18:58:47.940710 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:47.940614 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-wzdjn" event={"ID":"13d7fd32-6207-4178-ac83-914df4da7e9f","Type":"ContainerStarted","Data":"23868f7a0cc721a7b4fddd7753b1ad60ef3e76d666956d9fedadf994cbda581d"} Jul 07 18:58:47.941204 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:47.940740 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-wzdjn" Jul 07 18:58:47.942194 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:47.942166 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6d85fdd848-krbqg" event={"ID":"218f5aaa-287e-4243-ba90-a62a1912b472","Type":"ContainerStarted","Data":"6c8d15a8e0cc21ed0e3c395888a2659d1d09dc8e4113dce56acc0e78e8d2fd6d"} Jul 07 18:58:47.942305 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:47.942226 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6d85fdd848-krbqg" Jul 07 18:58:47.943597 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:47.943557 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-474mg" event={"ID":"c0fba180-a064-4905-86b3-df268d3d51c8","Type":"ContainerStarted","Data":"5e0886d3aa651775243f52ca5a58bdc424a8442eaf4c8f277668cc8401389421"} Jul 07 18:58:47.943675 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:47.943640 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-474mg" Jul 07 18:58:47.963080 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:47.963002 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-wzdjn" podStartSLOduration=1.216627443 podStartE2EDuration="2.962978569s" totalImagesPullingTime="1.746351126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:58:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:58:47.958168878 +0000 UTC m=+535.377529117" watchObservedRunningTime="2026-07-07 18:58:47.962978569 +0000 UTC m=+535.382338796" Jul 07 18:58:47.978837 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:47.978764 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-6d85fdd848-krbqg" podStartSLOduration=1.669940929 podStartE2EDuration="3.978741918s" totalImagesPullingTime="2.308800989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:58:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:58:47.976755978 +0000 UTC m=+535.396116206" watchObservedRunningTime="2026-07-07 18:58:47.978741918 +0000 UTC m=+535.398102145" Jul 07 18:58:47.996240 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:47.996194 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-474mg" podStartSLOduration=1.189566056 podStartE2EDuration="2.996177705s" totalImagesPullingTime="1.806611649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:58:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:58:47.994497367 +0000 UTC m=+535.413857591" watchObservedRunningTime="2026-07-07 18:58:47.996177705 +0000 UTC m=+535.415537937" Jul 07 18:58:53.954962 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:53.954926 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6d85fdd848-krbqg" Jul 07 18:58:58.950842 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:58.950737 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-474mg" Jul 07 18:58:58.950842 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:58:58.950821 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-wzdjn" Jul 07 18:59:21.647369 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.647305 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5"] Jul 07 18:59:21.651794 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.651775 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:21.654411 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.654388 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hpqdr\"" Jul 07 18:59:21.655412 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.655392 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 18:59:21.655412 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.655408 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 07 18:59:21.655638 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.655458 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 18:59:21.664166 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.664143 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5"] Jul 07 18:59:21.713063 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.713034 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tttn5\" (UID: \"53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:21.713201 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.713083 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tttn5\" (UID: \"53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:21.713201 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.713104 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tttn5\" (UID: \"53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:21.713201 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.713125 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg626\" (UniqueName: \"kubernetes.io/projected/53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e-kube-api-access-pg626\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tttn5\" (UID: \"53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:21.713201 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.713155 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tttn5\" (UID: \"53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:21.713337 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.713228 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tttn5\" (UID: \"53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:21.814290 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.814254 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tttn5\" (UID: \"53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:21.814290 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.814294 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tttn5\" (UID: \"53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:21.814501 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.814315 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg626\" (UniqueName: \"kubernetes.io/projected/53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e-kube-api-access-pg626\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tttn5\" (UID: \"53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:21.814501 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.814346 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tttn5\" (UID: \"53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:21.814501 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.814368 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tttn5\" (UID: \"53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:21.814501 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.814391 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tttn5\" (UID: \"53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:21.814789 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.814766 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tttn5\" (UID: \"53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:21.814863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.814806 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tttn5\" (UID: \"53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:21.814863 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.814835 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tttn5\" (UID: \"53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:21.816781 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.816758 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tttn5\" (UID: \"53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:21.817093 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.817072 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tttn5\" (UID: \"53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:21.823047 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.823024 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg626\" (UniqueName: \"kubernetes.io/projected/53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e-kube-api-access-pg626\") pod \"e2e-distinct-simulated-kserve-59975d69f9-tttn5\" (UID: \"53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:21.965611 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:21.965518 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:22.317976 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:22.317946 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5"] Jul 07 18:59:22.318564 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:59:22.318542 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53e2af7b_73b5_4ca5_8e4a_7efcbe8ebf5e.slice/crio-e7f81015eb5ce5cdd3ce3ab91d88f7f585c3f2769cae670f358b0d18879f9f33 WatchSource:0}: Error finding container e7f81015eb5ce5cdd3ce3ab91d88f7f585c3f2769cae670f358b0d18879f9f33: Status 404 returned error can't find the container with id e7f81015eb5ce5cdd3ce3ab91d88f7f585c3f2769cae670f358b0d18879f9f33 Jul 07 18:59:23.139969 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:23.139928 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" event={"ID":"53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e","Type":"ContainerStarted","Data":"e7f81015eb5ce5cdd3ce3ab91d88f7f585c3f2769cae670f358b0d18879f9f33"} Jul 07 18:59:24.147331 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:24.147293 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" event={"ID":"53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e","Type":"ContainerStarted","Data":"713cefd76be1120e9c15c4b28e8799f321cf2279c3e05b34f9e352227bf602cc"} Jul 07 18:59:24.166429 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:24.166380 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" podStartSLOduration=2.077635623 podStartE2EDuration="3.166364367s" totalImagesPullingTime="1.088728744s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:59:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:59:24.164706139 +0000 UTC m=+571.584066363" watchObservedRunningTime="2026-07-07 18:59:24.166364367 +0000 UTC m=+571.585724591" Jul 07 18:59:31.966428 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:31.966395 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:31.966955 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:31.966447 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:31.978550 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:31.978527 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:32.210238 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:32.210207 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-tttn5" Jul 07 18:59:39.711589 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.711544 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp"] Jul 07 18:59:39.715338 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.715319 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:39.717849 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.717827 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 07 18:59:39.723961 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.723935 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp"] Jul 07 18:59:39.864731 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.864697 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9w2m\" (UniqueName: \"kubernetes.io/projected/315c54b5-0d92-4893-b3e4-c08f079befd3-kube-api-access-r9w2m\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp\" (UID: \"315c54b5-0d92-4893-b3e4-c08f079befd3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:39.864889 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.864738 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/315c54b5-0d92-4893-b3e4-c08f079befd3-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp\" (UID: \"315c54b5-0d92-4893-b3e4-c08f079befd3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:39.864889 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.864759 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/315c54b5-0d92-4893-b3e4-c08f079befd3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp\" (UID: \"315c54b5-0d92-4893-b3e4-c08f079befd3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:39.864889 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.864816 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/315c54b5-0d92-4893-b3e4-c08f079befd3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp\" (UID: \"315c54b5-0d92-4893-b3e4-c08f079befd3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:39.864889 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.864848 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/315c54b5-0d92-4893-b3e4-c08f079befd3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp\" (UID: \"315c54b5-0d92-4893-b3e4-c08f079befd3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:39.865028 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.864921 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/315c54b5-0d92-4893-b3e4-c08f079befd3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp\" (UID: \"315c54b5-0d92-4893-b3e4-c08f079befd3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:39.966128 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.966046 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/315c54b5-0d92-4893-b3e4-c08f079befd3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp\" (UID: \"315c54b5-0d92-4893-b3e4-c08f079befd3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:39.966128 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.966113 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9w2m\" (UniqueName: \"kubernetes.io/projected/315c54b5-0d92-4893-b3e4-c08f079befd3-kube-api-access-r9w2m\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp\" (UID: \"315c54b5-0d92-4893-b3e4-c08f079befd3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:39.966349 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.966151 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/315c54b5-0d92-4893-b3e4-c08f079befd3-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp\" (UID: \"315c54b5-0d92-4893-b3e4-c08f079befd3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:39.966349 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.966176 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/315c54b5-0d92-4893-b3e4-c08f079befd3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp\" (UID: \"315c54b5-0d92-4893-b3e4-c08f079befd3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:39.966349 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.966223 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/315c54b5-0d92-4893-b3e4-c08f079befd3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp\" (UID: \"315c54b5-0d92-4893-b3e4-c08f079befd3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:39.966349 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.966269 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/315c54b5-0d92-4893-b3e4-c08f079befd3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp\" (UID: \"315c54b5-0d92-4893-b3e4-c08f079befd3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:39.966588 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.966499 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/315c54b5-0d92-4893-b3e4-c08f079befd3-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp\" (UID: \"315c54b5-0d92-4893-b3e4-c08f079befd3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:39.966657 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.966617 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/315c54b5-0d92-4893-b3e4-c08f079befd3-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp\" (UID: \"315c54b5-0d92-4893-b3e4-c08f079befd3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:39.966657 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.966643 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/315c54b5-0d92-4893-b3e4-c08f079befd3-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp\" (UID: \"315c54b5-0d92-4893-b3e4-c08f079befd3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:39.968611 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.968558 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/315c54b5-0d92-4893-b3e4-c08f079befd3-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp\" (UID: \"315c54b5-0d92-4893-b3e4-c08f079befd3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:39.968786 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.968766 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/315c54b5-0d92-4893-b3e4-c08f079befd3-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp\" (UID: \"315c54b5-0d92-4893-b3e4-c08f079befd3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:39.974869 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:39.974848 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9w2m\" (UniqueName: \"kubernetes.io/projected/315c54b5-0d92-4893-b3e4-c08f079befd3-kube-api-access-r9w2m\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp\" (UID: \"315c54b5-0d92-4893-b3e4-c08f079befd3\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:40.026814 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:40.026787 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:40.372266 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:40.372244 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp"] Jul 07 18:59:40.373759 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:59:40.373729 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod315c54b5_0d92_4893_b3e4_c08f079befd3.slice/crio-0777058dfded57ee8ff03a382acf60f39c8717a90aaa61da439bc95016ae2bfa WatchSource:0}: Error finding container 0777058dfded57ee8ff03a382acf60f39c8717a90aaa61da439bc95016ae2bfa: Status 404 returned error can't find the container with id 0777058dfded57ee8ff03a382acf60f39c8717a90aaa61da439bc95016ae2bfa Jul 07 18:59:41.251735 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:41.251704 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" event={"ID":"315c54b5-0d92-4893-b3e4-c08f079befd3","Type":"ContainerStarted","Data":"58e64e12ba9bd7dd53c38dffe075c2f478c3cd8837715c0560a210e9ad8601bf"} Jul 07 18:59:41.252172 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:41.251754 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" event={"ID":"315c54b5-0d92-4893-b3e4-c08f079befd3","Type":"ContainerStarted","Data":"0777058dfded57ee8ff03a382acf60f39c8717a90aaa61da439bc95016ae2bfa"} Jul 07 18:59:41.275864 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:41.275821 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" podStartSLOduration=2.109300853 podStartE2EDuration="2.275802726s" totalImagesPullingTime="166.501873ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:59:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:59:41.273396102 +0000 UTC m=+588.692756350" watchObservedRunningTime="2026-07-07 18:59:41.275802726 +0000 UTC m=+588.695162952" Jul 07 18:59:43.617673 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.617640 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk"] Jul 07 18:59:43.621531 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.621512 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:43.624060 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.624041 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 07 18:59:43.632277 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.632251 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk"] Jul 07 18:59:43.696739 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.696708 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a9eb5d0-823f-43d8-8e5e-5f74651dc753-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hwjk\" (UID: \"3a9eb5d0-823f-43d8-8e5e-5f74651dc753\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:43.696739 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.696744 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a9eb5d0-823f-43d8-8e5e-5f74651dc753-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hwjk\" (UID: \"3a9eb5d0-823f-43d8-8e5e-5f74651dc753\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:43.696919 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.696777 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a9eb5d0-823f-43d8-8e5e-5f74651dc753-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hwjk\" (UID: \"3a9eb5d0-823f-43d8-8e5e-5f74651dc753\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:43.696919 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.696819 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c56cj\" (UniqueName: \"kubernetes.io/projected/3a9eb5d0-823f-43d8-8e5e-5f74651dc753-kube-api-access-c56cj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hwjk\" (UID: \"3a9eb5d0-823f-43d8-8e5e-5f74651dc753\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:43.696919 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.696899 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a9eb5d0-823f-43d8-8e5e-5f74651dc753-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hwjk\" (UID: \"3a9eb5d0-823f-43d8-8e5e-5f74651dc753\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:43.697015 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.696922 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a9eb5d0-823f-43d8-8e5e-5f74651dc753-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hwjk\" (UID: \"3a9eb5d0-823f-43d8-8e5e-5f74651dc753\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:43.797639 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.797606 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a9eb5d0-823f-43d8-8e5e-5f74651dc753-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hwjk\" (UID: \"3a9eb5d0-823f-43d8-8e5e-5f74651dc753\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:43.797825 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.797656 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a9eb5d0-823f-43d8-8e5e-5f74651dc753-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hwjk\" (UID: \"3a9eb5d0-823f-43d8-8e5e-5f74651dc753\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:43.797825 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.797713 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a9eb5d0-823f-43d8-8e5e-5f74651dc753-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hwjk\" (UID: \"3a9eb5d0-823f-43d8-8e5e-5f74651dc753\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:43.797957 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.797844 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a9eb5d0-823f-43d8-8e5e-5f74651dc753-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hwjk\" (UID: \"3a9eb5d0-823f-43d8-8e5e-5f74651dc753\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:43.797957 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.797901 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a9eb5d0-823f-43d8-8e5e-5f74651dc753-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hwjk\" (UID: \"3a9eb5d0-823f-43d8-8e5e-5f74651dc753\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:43.797957 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.797928 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c56cj\" (UniqueName: \"kubernetes.io/projected/3a9eb5d0-823f-43d8-8e5e-5f74651dc753-kube-api-access-c56cj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hwjk\" (UID: \"3a9eb5d0-823f-43d8-8e5e-5f74651dc753\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:43.798162 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.797998 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3a9eb5d0-823f-43d8-8e5e-5f74651dc753-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hwjk\" (UID: \"3a9eb5d0-823f-43d8-8e5e-5f74651dc753\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:43.798219 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.798157 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a9eb5d0-823f-43d8-8e5e-5f74651dc753-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hwjk\" (UID: \"3a9eb5d0-823f-43d8-8e5e-5f74651dc753\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:43.798344 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.798319 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3a9eb5d0-823f-43d8-8e5e-5f74651dc753-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hwjk\" (UID: \"3a9eb5d0-823f-43d8-8e5e-5f74651dc753\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:43.800132 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.800111 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3a9eb5d0-823f-43d8-8e5e-5f74651dc753-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hwjk\" (UID: \"3a9eb5d0-823f-43d8-8e5e-5f74651dc753\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:43.800632 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.800613 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3a9eb5d0-823f-43d8-8e5e-5f74651dc753-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hwjk\" (UID: \"3a9eb5d0-823f-43d8-8e5e-5f74651dc753\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:43.806798 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.806775 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c56cj\" (UniqueName: \"kubernetes.io/projected/3a9eb5d0-823f-43d8-8e5e-5f74651dc753-kube-api-access-c56cj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-6hwjk\" (UID: \"3a9eb5d0-823f-43d8-8e5e-5f74651dc753\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:43.933248 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:43.933175 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:44.302491 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:44.302463 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk"] Jul 07 18:59:44.304195 ip-10-0-142-66 kubenswrapper[2564]: W0707 18:59:44.304167 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a9eb5d0_823f_43d8_8e5e_5f74651dc753.slice/crio-42a5cd61f0940ba3c85decd38ccee54a21343bf4851eb5b8c5815e403078cefa WatchSource:0}: Error finding container 42a5cd61f0940ba3c85decd38ccee54a21343bf4851eb5b8c5815e403078cefa: Status 404 returned error can't find the container with id 42a5cd61f0940ba3c85decd38ccee54a21343bf4851eb5b8c5815e403078cefa Jul 07 18:59:45.276953 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:45.276917 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" event={"ID":"3a9eb5d0-823f-43d8-8e5e-5f74651dc753","Type":"ContainerStarted","Data":"97a83c7fc032bf30f3200d89c7f19e4138b5d8bd55b33bea05dfcd41b0dd3ed4"} Jul 07 18:59:45.277335 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:45.276972 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" event={"ID":"3a9eb5d0-823f-43d8-8e5e-5f74651dc753","Type":"ContainerStarted","Data":"42a5cd61f0940ba3c85decd38ccee54a21343bf4851eb5b8c5815e403078cefa"} Jul 07 18:59:45.295181 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:45.295131 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" podStartSLOduration=2.127231194 podStartE2EDuration="2.295114603s" totalImagesPullingTime="167.883409ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:59:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:59:45.293850017 +0000 UTC m=+592.713210244" watchObservedRunningTime="2026-07-07 18:59:45.295114603 +0000 UTC m=+592.714474881" Jul 07 18:59:50.027943 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:50.027908 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:50.028336 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:50.027966 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:50.040292 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:50.040263 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:50.318016 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:50.317939 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp" Jul 07 18:59:53.050607 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:53.050561 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/1.log" Jul 07 18:59:53.051031 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:53.050675 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/1.log" Jul 07 18:59:53.933397 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:53.933365 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:53.935617 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:53.933420 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:53.945983 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:53.945962 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 18:59:54.341351 ip-10-0-142-66 kubenswrapper[2564]: I0707 18:59:54.341323 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-6hwjk" Jul 07 19:02:18.552681 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:18.552602 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d8f857c75-6xhvw"] Jul 07 19:02:18.553259 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:18.552975 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-d8f857c75-6xhvw" podUID="2ad58a61-cbb9-4497-994b-e7009ed93a80" containerName="manager" containerID="cri-o://b754d13e9a4189cb9497afbf78130650ce6de1ed41c842196abe61519309b5a7" gracePeriod=30 Jul 07 19:02:18.913914 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:18.913890 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d8f857c75-6xhvw" Jul 07 19:02:19.034587 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:19.034554 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqjk5\" (UniqueName: \"kubernetes.io/projected/2ad58a61-cbb9-4497-994b-e7009ed93a80-kube-api-access-wqjk5\") pod \"2ad58a61-cbb9-4497-994b-e7009ed93a80\" (UID: \"2ad58a61-cbb9-4497-994b-e7009ed93a80\") " Jul 07 19:02:19.034749 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:19.034677 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ad58a61-cbb9-4497-994b-e7009ed93a80-webhook-cert\") pod \"2ad58a61-cbb9-4497-994b-e7009ed93a80\" (UID: \"2ad58a61-cbb9-4497-994b-e7009ed93a80\") " Jul 07 19:02:19.036696 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:19.036664 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ad58a61-cbb9-4497-994b-e7009ed93a80-kube-api-access-wqjk5" (OuterVolumeSpecName: "kube-api-access-wqjk5") pod "2ad58a61-cbb9-4497-994b-e7009ed93a80" (UID: "2ad58a61-cbb9-4497-994b-e7009ed93a80"). InnerVolumeSpecName "kube-api-access-wqjk5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:02:19.036696 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:19.036675 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ad58a61-cbb9-4497-994b-e7009ed93a80-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "2ad58a61-cbb9-4497-994b-e7009ed93a80" (UID: "2ad58a61-cbb9-4497-994b-e7009ed93a80"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:02:19.135280 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:19.135254 2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ad58a61-cbb9-4497-994b-e7009ed93a80-webhook-cert\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 19:02:19.135280 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:19.135277 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqjk5\" (UniqueName: \"kubernetes.io/projected/2ad58a61-cbb9-4497-994b-e7009ed93a80-kube-api-access-wqjk5\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 19:02:19.176249 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:19.176180 2564 generic.go:356] "Generic (PLEG): container finished" podID="2ad58a61-cbb9-4497-994b-e7009ed93a80" containerID="b754d13e9a4189cb9497afbf78130650ce6de1ed41c842196abe61519309b5a7" exitCode=0 Jul 07 19:02:19.176249 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:19.176231 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d8f857c75-6xhvw" Jul 07 19:02:19.176249 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:19.176233 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d8f857c75-6xhvw" event={"ID":"2ad58a61-cbb9-4497-994b-e7009ed93a80","Type":"ContainerDied","Data":"b754d13e9a4189cb9497afbf78130650ce6de1ed41c842196abe61519309b5a7"} Jul 07 19:02:19.176445 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:19.176289 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d8f857c75-6xhvw" event={"ID":"2ad58a61-cbb9-4497-994b-e7009ed93a80","Type":"ContainerDied","Data":"f970e050c17773a52a7697586b483096aaab43d36c3d718f6e4f65a4d5e8c709"} Jul 07 19:02:19.176445 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:19.176312 2564 scope.go:117] "RemoveContainer" containerID="b754d13e9a4189cb9497afbf78130650ce6de1ed41c842196abe61519309b5a7" Jul 07 19:02:19.185271 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:19.185254 2564 scope.go:117] "RemoveContainer" containerID="b754d13e9a4189cb9497afbf78130650ce6de1ed41c842196abe61519309b5a7" Jul 07 19:02:19.185517 ip-10-0-142-66 kubenswrapper[2564]: E0707 19:02:19.185500 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b754d13e9a4189cb9497afbf78130650ce6de1ed41c842196abe61519309b5a7\": container with ID starting with b754d13e9a4189cb9497afbf78130650ce6de1ed41c842196abe61519309b5a7 not found: ID does not exist" containerID="b754d13e9a4189cb9497afbf78130650ce6de1ed41c842196abe61519309b5a7" Jul 07 19:02:19.185569 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:19.185527 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b754d13e9a4189cb9497afbf78130650ce6de1ed41c842196abe61519309b5a7"} err="failed to get container status \"b754d13e9a4189cb9497afbf78130650ce6de1ed41c842196abe61519309b5a7\": rpc error: code = NotFound desc = could not find container \"b754d13e9a4189cb9497afbf78130650ce6de1ed41c842196abe61519309b5a7\": container with ID starting with b754d13e9a4189cb9497afbf78130650ce6de1ed41c842196abe61519309b5a7 not found: ID does not exist" Jul 07 19:02:19.195927 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:19.195905 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d8f857c75-6xhvw"] Jul 07 19:02:19.200839 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:19.200818 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d8f857c75-6xhvw"] Jul 07 19:02:19.994518 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:19.994486 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d8f857c75-9w7hs"] Jul 07 19:02:19.995064 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:19.995045 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ad58a61-cbb9-4497-994b-e7009ed93a80" containerName="manager" Jul 07 19:02:19.995108 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:19.995067 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ad58a61-cbb9-4497-994b-e7009ed93a80" containerName="manager" Jul 07 19:02:19.995140 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:19.995133 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ad58a61-cbb9-4497-994b-e7009ed93a80" containerName="manager" Jul 07 19:02:19.999470 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:19.999455 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d8f857c75-9w7hs" Jul 07 19:02:20.002035 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:20.002013 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 19:02:20.002154 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:20.002014 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-w7tt9\"" Jul 07 19:02:20.002154 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:20.002016 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 19:02:20.010836 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:20.010816 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d8f857c75-9w7hs"] Jul 07 19:02:20.042998 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:20.042973 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvb9f\" (UniqueName: \"kubernetes.io/projected/074b8f0e-6b1b-4c33-a595-75f1fa7d3da3-kube-api-access-xvb9f\") pod \"maas-controller-d8f857c75-9w7hs\" (UID: \"074b8f0e-6b1b-4c33-a595-75f1fa7d3da3\") " pod="opendatahub/maas-controller-d8f857c75-9w7hs" Jul 07 19:02:20.043109 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:20.043037 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/074b8f0e-6b1b-4c33-a595-75f1fa7d3da3-webhook-cert\") pod \"maas-controller-d8f857c75-9w7hs\" (UID: \"074b8f0e-6b1b-4c33-a595-75f1fa7d3da3\") " pod="opendatahub/maas-controller-d8f857c75-9w7hs" Jul 07 19:02:20.143875 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:20.143847 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvb9f\" (UniqueName: \"kubernetes.io/projected/074b8f0e-6b1b-4c33-a595-75f1fa7d3da3-kube-api-access-xvb9f\") pod \"maas-controller-d8f857c75-9w7hs\" (UID: \"074b8f0e-6b1b-4c33-a595-75f1fa7d3da3\") " pod="opendatahub/maas-controller-d8f857c75-9w7hs" Jul 07 19:02:20.144008 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:20.143899 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/074b8f0e-6b1b-4c33-a595-75f1fa7d3da3-webhook-cert\") pod \"maas-controller-d8f857c75-9w7hs\" (UID: \"074b8f0e-6b1b-4c33-a595-75f1fa7d3da3\") " pod="opendatahub/maas-controller-d8f857c75-9w7hs" Jul 07 19:02:20.146357 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:20.146329 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/074b8f0e-6b1b-4c33-a595-75f1fa7d3da3-webhook-cert\") pod \"maas-controller-d8f857c75-9w7hs\" (UID: \"074b8f0e-6b1b-4c33-a595-75f1fa7d3da3\") " pod="opendatahub/maas-controller-d8f857c75-9w7hs" Jul 07 19:02:20.153489 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:20.153465 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvb9f\" (UniqueName: \"kubernetes.io/projected/074b8f0e-6b1b-4c33-a595-75f1fa7d3da3-kube-api-access-xvb9f\") pod \"maas-controller-d8f857c75-9w7hs\" (UID: \"074b8f0e-6b1b-4c33-a595-75f1fa7d3da3\") " pod="opendatahub/maas-controller-d8f857c75-9w7hs" Jul 07 19:02:20.310353 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:20.310323 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d8f857c75-9w7hs" Jul 07 19:02:20.658401 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:20.658359 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d8f857c75-9w7hs"] Jul 07 19:02:20.658642 ip-10-0-142-66 kubenswrapper[2564]: W0707 19:02:20.658618 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod074b8f0e_6b1b_4c33_a595_75f1fa7d3da3.slice/crio-0408d6094d6995d7f0f76430d08610de3d51b4bf2336dc6aa93635c1adc67cf8 WatchSource:0}: Error finding container 0408d6094d6995d7f0f76430d08610de3d51b4bf2336dc6aa93635c1adc67cf8: Status 404 returned error can't find the container with id 0408d6094d6995d7f0f76430d08610de3d51b4bf2336dc6aa93635c1adc67cf8 Jul 07 19:02:20.660006 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:20.659986 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 19:02:21.105556 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:21.105527 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ad58a61-cbb9-4497-994b-e7009ed93a80" path="/var/lib/kubelet/pods/2ad58a61-cbb9-4497-994b-e7009ed93a80/volumes" Jul 07 19:02:21.189793 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:21.189768 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d8f857c75-9w7hs" event={"ID":"074b8f0e-6b1b-4c33-a595-75f1fa7d3da3","Type":"ContainerStarted","Data":"0408d6094d6995d7f0f76430d08610de3d51b4bf2336dc6aa93635c1adc67cf8"} Jul 07 19:02:22.197752 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:22.197720 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d8f857c75-9w7hs" event={"ID":"074b8f0e-6b1b-4c33-a595-75f1fa7d3da3","Type":"ContainerStarted","Data":"5e4f0d5eb72758d396aa5484e3653aa08d7af6d5208c1226f75dd5de048c6670"} Jul 07 19:02:22.198238 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:22.197854 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d8f857c75-9w7hs" Jul 07 19:02:22.215922 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:22.215873 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d8f857c75-9w7hs" podStartSLOduration=2.691951646 podStartE2EDuration="3.215854974s" totalImagesPullingTime="523.903328ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:02:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:02:22.215290597 +0000 UTC m=+749.634650834" watchObservedRunningTime="2026-07-07 19:02:22.215854974 +0000 UTC m=+749.635215200" Jul 07 19:02:33.209838 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:02:33.209808 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d8f857c75-9w7hs" Jul 07 19:04:53.089149 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:04:53.089120 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/1.log" Jul 07 19:04:53.093537 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:04:53.093513 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/1.log" Jul 07 19:07:06.462464 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:07:06.462430 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259"] Jul 07 19:07:06.462984 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:07:06.462685 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259" podUID="dfce2c8e-9a17-49b5-92a3-9469dabd0a2f" containerName="manager" containerID="cri-o://6e7a3ee990b47f29b4e3e4ba596b2f85b1105415f76c80296f60668a7d0e12b8" gracePeriod=10 Jul 07 19:07:09.501312 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:07:09.501288 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259" Jul 07 19:07:09.597701 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:07:09.597630 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbfvc\" (UniqueName: \"kubernetes.io/projected/dfce2c8e-9a17-49b5-92a3-9469dabd0a2f-kube-api-access-pbfvc\") pod \"dfce2c8e-9a17-49b5-92a3-9469dabd0a2f\" (UID: \"dfce2c8e-9a17-49b5-92a3-9469dabd0a2f\") " Jul 07 19:07:09.597838 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:07:09.597801 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfce2c8e-9a17-49b5-92a3-9469dabd0a2f-extensions-socket-volume\") pod \"dfce2c8e-9a17-49b5-92a3-9469dabd0a2f\" (UID: \"dfce2c8e-9a17-49b5-92a3-9469dabd0a2f\") " Jul 07 19:07:09.598153 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:07:09.598122 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfce2c8e-9a17-49b5-92a3-9469dabd0a2f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dfce2c8e-9a17-49b5-92a3-9469dabd0a2f" (UID: "dfce2c8e-9a17-49b5-92a3-9469dabd0a2f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:07:09.599762 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:07:09.599744 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfce2c8e-9a17-49b5-92a3-9469dabd0a2f-kube-api-access-pbfvc" (OuterVolumeSpecName: "kube-api-access-pbfvc") pod "dfce2c8e-9a17-49b5-92a3-9469dabd0a2f" (UID: "dfce2c8e-9a17-49b5-92a3-9469dabd0a2f"). InnerVolumeSpecName "kube-api-access-pbfvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:07:09.699368 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:07:09.699346 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dfce2c8e-9a17-49b5-92a3-9469dabd0a2f-extensions-socket-volume\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 19:07:09.699368 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:07:09.699367 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbfvc\" (UniqueName: \"kubernetes.io/projected/dfce2c8e-9a17-49b5-92a3-9469dabd0a2f-kube-api-access-pbfvc\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 19:07:09.858543 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:07:09.858482 2564 generic.go:356] "Generic (PLEG): container finished" podID="dfce2c8e-9a17-49b5-92a3-9469dabd0a2f" containerID="6e7a3ee990b47f29b4e3e4ba596b2f85b1105415f76c80296f60668a7d0e12b8" exitCode=0 Jul 07 19:07:09.858543 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:07:09.858528 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259" Jul 07 19:07:09.858722 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:07:09.858590 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259" event={"ID":"dfce2c8e-9a17-49b5-92a3-9469dabd0a2f","Type":"ContainerDied","Data":"6e7a3ee990b47f29b4e3e4ba596b2f85b1105415f76c80296f60668a7d0e12b8"} Jul 07 19:07:09.858722 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:07:09.858651 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259" event={"ID":"dfce2c8e-9a17-49b5-92a3-9469dabd0a2f","Type":"ContainerDied","Data":"c6e48f683191c42958e009ce24a74d00badc7794b4c3dcfd0db2a39b701407ec"} Jul 07 19:07:09.858722 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:07:09.858680 2564 scope.go:117] "RemoveContainer" containerID="6e7a3ee990b47f29b4e3e4ba596b2f85b1105415f76c80296f60668a7d0e12b8" Jul 07 19:07:09.869094 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:07:09.869078 2564 scope.go:117] "RemoveContainer" containerID="6e7a3ee990b47f29b4e3e4ba596b2f85b1105415f76c80296f60668a7d0e12b8" Jul 07 19:07:09.869337 ip-10-0-142-66 kubenswrapper[2564]: E0707 19:07:09.869319 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e7a3ee990b47f29b4e3e4ba596b2f85b1105415f76c80296f60668a7d0e12b8\": container with ID starting with 6e7a3ee990b47f29b4e3e4ba596b2f85b1105415f76c80296f60668a7d0e12b8 not found: ID does not exist" containerID="6e7a3ee990b47f29b4e3e4ba596b2f85b1105415f76c80296f60668a7d0e12b8" Jul 07 19:07:09.869391 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:07:09.869350 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e7a3ee990b47f29b4e3e4ba596b2f85b1105415f76c80296f60668a7d0e12b8"} err="failed to get container status \"6e7a3ee990b47f29b4e3e4ba596b2f85b1105415f76c80296f60668a7d0e12b8\": rpc error: code = NotFound desc = could not find container \"6e7a3ee990b47f29b4e3e4ba596b2f85b1105415f76c80296f60668a7d0e12b8\": container with ID starting with 6e7a3ee990b47f29b4e3e4ba596b2f85b1105415f76c80296f60668a7d0e12b8 not found: ID does not exist" Jul 07 19:07:09.885929 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:07:09.885908 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259"] Jul 07 19:07:09.890402 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:07:09.890380 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bq259"] Jul 07 19:07:11.108162 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:07:11.108129 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfce2c8e-9a17-49b5-92a3-9469dabd0a2f" path="/var/lib/kubelet/pods/dfce2c8e-9a17-49b5-92a3-9469dabd0a2f/volumes" Jul 07 19:08:07.712298 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:08:07.712263 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5sggt"] Jul 07 19:08:07.712925 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:08:07.712908 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfce2c8e-9a17-49b5-92a3-9469dabd0a2f" containerName="manager" Jul 07 19:08:07.712976 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:08:07.712927 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfce2c8e-9a17-49b5-92a3-9469dabd0a2f" containerName="manager" Jul 07 19:08:07.713031 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:08:07.713022 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfce2c8e-9a17-49b5-92a3-9469dabd0a2f" containerName="manager" Jul 07 19:08:07.716033 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:08:07.716019 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5sggt" Jul 07 19:08:07.719354 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:08:07.719334 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4l2bz\"" Jul 07 19:08:07.731676 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:08:07.731651 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5sggt"] Jul 07 19:08:07.896716 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:08:07.896687 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9qph\" (UniqueName: \"kubernetes.io/projected/07ab7460-9b7c-4304-bfc0-5b9d7c704746-kube-api-access-n9qph\") pod \"kuadrant-operator-controller-manager-86878c769f-5sggt\" (UID: \"07ab7460-9b7c-4304-bfc0-5b9d7c704746\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5sggt" Jul 07 19:08:07.896881 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:08:07.896798 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/07ab7460-9b7c-4304-bfc0-5b9d7c704746-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5sggt\" (UID: \"07ab7460-9b7c-4304-bfc0-5b9d7c704746\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5sggt" Jul 07 19:08:07.997497 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:08:07.997471 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/07ab7460-9b7c-4304-bfc0-5b9d7c704746-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5sggt\" (UID: \"07ab7460-9b7c-4304-bfc0-5b9d7c704746\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5sggt" Jul 07 19:08:07.997646 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:08:07.997519 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9qph\" (UniqueName: \"kubernetes.io/projected/07ab7460-9b7c-4304-bfc0-5b9d7c704746-kube-api-access-n9qph\") pod \"kuadrant-operator-controller-manager-86878c769f-5sggt\" (UID: \"07ab7460-9b7c-4304-bfc0-5b9d7c704746\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5sggt" Jul 07 19:08:07.997857 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:08:07.997840 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/07ab7460-9b7c-4304-bfc0-5b9d7c704746-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5sggt\" (UID: \"07ab7460-9b7c-4304-bfc0-5b9d7c704746\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5sggt" Jul 07 19:08:08.009788 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:08:08.009761 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9qph\" (UniqueName: \"kubernetes.io/projected/07ab7460-9b7c-4304-bfc0-5b9d7c704746-kube-api-access-n9qph\") pod \"kuadrant-operator-controller-manager-86878c769f-5sggt\" (UID: \"07ab7460-9b7c-4304-bfc0-5b9d7c704746\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5sggt" Jul 07 19:08:08.026409 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:08:08.026384 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5sggt" Jul 07 19:08:08.389533 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:08:08.389509 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5sggt"] Jul 07 19:08:08.390335 ip-10-0-142-66 kubenswrapper[2564]: W0707 19:08:08.390315 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07ab7460_9b7c_4304_bfc0_5b9d7c704746.slice/crio-33cb621c576dc489688357b1ae659a9452e92fb7f6190f6a3089f6e33431d68f WatchSource:0}: Error finding container 33cb621c576dc489688357b1ae659a9452e92fb7f6190f6a3089f6e33431d68f: Status 404 returned error can't find the container with id 33cb621c576dc489688357b1ae659a9452e92fb7f6190f6a3089f6e33431d68f Jul 07 19:08:09.202826 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:08:09.202784 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5sggt" event={"ID":"07ab7460-9b7c-4304-bfc0-5b9d7c704746","Type":"ContainerStarted","Data":"e37146b90dead46ca73c4306a52ccfae92cd0b2e99251c3f6fb094fffc861b2a"} Jul 07 19:08:09.203209 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:08:09.202951 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5sggt" Jul 07 19:08:09.203209 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:08:09.202969 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5sggt" event={"ID":"07ab7460-9b7c-4304-bfc0-5b9d7c704746","Type":"ContainerStarted","Data":"33cb621c576dc489688357b1ae659a9452e92fb7f6190f6a3089f6e33431d68f"} Jul 07 19:08:09.226524 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:08:09.226460 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5sggt" podStartSLOduration=2.226443238 podStartE2EDuration="2.226443238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:08:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:08:09.225026377 +0000 UTC m=+1096.644386602" watchObservedRunningTime="2026-07-07 19:08:09.226443238 +0000 UTC m=+1096.645803463" Jul 07 19:08:20.210916 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:08:20.210879 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5sggt" Jul 07 19:09:53.127355 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:09:53.127326 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/1.log" Jul 07 19:09:53.134570 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:09:53.134545 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/1.log" Jul 07 19:14:53.175984 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:14:53.175961 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/1.log" Jul 07 19:14:53.185324 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:14:53.185297 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/1.log" Jul 07 19:15:00.844702 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:00.844664 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl"] Jul 07 19:15:00.848966 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:00.848937 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" Jul 07 19:15:00.851630 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:00.851603 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-914b2894-serving-cert\"" Jul 07 19:15:00.851770 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:00.851702 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-hxp29\"" Jul 07 19:15:00.863936 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:00.863909 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl"] Jul 07 19:15:01.005819 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:01.005781 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/430a211d-9ae5-43e6-9725-0bc75de82a9b-openshift-service-ca\") pod \"maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl\" (UID: \"430a211d-9ae5-43e6-9725-0bc75de82a9b\") " pod="opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" Jul 07 19:15:01.006005 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:01.005838 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/430a211d-9ae5-43e6-9725-0bc75de82a9b-maas-api-tls\") pod \"maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl\" (UID: \"430a211d-9ae5-43e6-9725-0bc75de82a9b\") " pod="opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" Jul 07 19:15:01.006005 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:01.005892 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87674\" (UniqueName: \"kubernetes.io/projected/430a211d-9ae5-43e6-9725-0bc75de82a9b-kube-api-access-87674\") pod \"maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl\" (UID: \"430a211d-9ae5-43e6-9725-0bc75de82a9b\") " pod="opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" Jul 07 19:15:01.107127 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:01.107043 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87674\" (UniqueName: \"kubernetes.io/projected/430a211d-9ae5-43e6-9725-0bc75de82a9b-kube-api-access-87674\") pod \"maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl\" (UID: \"430a211d-9ae5-43e6-9725-0bc75de82a9b\") " pod="opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" Jul 07 19:15:01.107303 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:01.107129 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/430a211d-9ae5-43e6-9725-0bc75de82a9b-openshift-service-ca\") pod \"maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl\" (UID: \"430a211d-9ae5-43e6-9725-0bc75de82a9b\") " pod="opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" Jul 07 19:15:01.107303 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:01.107168 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/430a211d-9ae5-43e6-9725-0bc75de82a9b-maas-api-tls\") pod \"maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl\" (UID: \"430a211d-9ae5-43e6-9725-0bc75de82a9b\") " pod="opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" Jul 07 19:15:01.107303 ip-10-0-142-66 kubenswrapper[2564]: E0707 19:15:01.107265 2564 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-914b2894-serving-cert: secret "maas-api-e2e-derive-914b2894-serving-cert" not found Jul 07 19:15:01.107475 ip-10-0-142-66 kubenswrapper[2564]: E0707 19:15:01.107311 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/430a211d-9ae5-43e6-9725-0bc75de82a9b-maas-api-tls podName:430a211d-9ae5-43e6-9725-0bc75de82a9b nodeName:}" failed. No retries permitted until 2026-07-07 19:15:01.607295463 +0000 UTC m=+1509.026655666 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/430a211d-9ae5-43e6-9725-0bc75de82a9b-maas-api-tls") pod "maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" (UID: "430a211d-9ae5-43e6-9725-0bc75de82a9b") : secret "maas-api-e2e-derive-914b2894-serving-cert" not found Jul 07 19:15:01.107751 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:01.107733 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/430a211d-9ae5-43e6-9725-0bc75de82a9b-openshift-service-ca\") pod \"maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl\" (UID: \"430a211d-9ae5-43e6-9725-0bc75de82a9b\") " pod="opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" Jul 07 19:15:01.116936 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:01.116910 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87674\" (UniqueName: \"kubernetes.io/projected/430a211d-9ae5-43e6-9725-0bc75de82a9b-kube-api-access-87674\") pod \"maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl\" (UID: \"430a211d-9ae5-43e6-9725-0bc75de82a9b\") " pod="opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" Jul 07 19:15:01.612312 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:01.612279 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/430a211d-9ae5-43e6-9725-0bc75de82a9b-maas-api-tls\") pod \"maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl\" (UID: \"430a211d-9ae5-43e6-9725-0bc75de82a9b\") " pod="opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" Jul 07 19:15:01.614963 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:01.614944 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/430a211d-9ae5-43e6-9725-0bc75de82a9b-maas-api-tls\") pod \"maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl\" (UID: \"430a211d-9ae5-43e6-9725-0bc75de82a9b\") " pod="opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" Jul 07 19:15:01.761345 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:01.761305 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" Jul 07 19:15:02.115963 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:02.115941 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl"] Jul 07 19:15:02.118625 ip-10-0-142-66 kubenswrapper[2564]: W0707 19:15:02.118557 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod430a211d_9ae5_43e6_9725_0bc75de82a9b.slice/crio-d964a8af6786fc1b8a6585775a94444cc3ea4766f9e5acf558b50b51eb840fb5 WatchSource:0}: Error finding container d964a8af6786fc1b8a6585775a94444cc3ea4766f9e5acf558b50b51eb840fb5: Status 404 returned error can't find the container with id d964a8af6786fc1b8a6585775a94444cc3ea4766f9e5acf558b50b51eb840fb5 Jul 07 19:15:02.119814 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:02.119795 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 19:15:02.612165 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:02.612130 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" event={"ID":"430a211d-9ae5-43e6-9725-0bc75de82a9b","Type":"ContainerStarted","Data":"d964a8af6786fc1b8a6585775a94444cc3ea4766f9e5acf558b50b51eb840fb5"} Jul 07 19:15:03.619730 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:03.619687 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" event={"ID":"430a211d-9ae5-43e6-9725-0bc75de82a9b","Type":"ContainerStarted","Data":"95c5c2fe11a9959483bdd017e6f07ffc50a128967382058df256578aec56a3f5"} Jul 07 19:15:03.620113 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:03.619850 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" Jul 07 19:15:03.638557 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:03.638499 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" podStartSLOduration=3.169211661 podStartE2EDuration="3.638480985s" totalImagesPullingTime="469.269324ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:15:03.637373268 +0000 UTC m=+1511.056733495" watchObservedRunningTime="2026-07-07 19:15:03.638480985 +0000 UTC m=+1511.057841211" Jul 07 19:15:05.514479 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:05.514446 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl"] Jul 07 19:15:05.634029 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:05.633986 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" podUID="430a211d-9ae5-43e6-9725-0bc75de82a9b" containerName="maas-api" containerID="cri-o://95c5c2fe11a9959483bdd017e6f07ffc50a128967382058df256578aec56a3f5" gracePeriod=30 Jul 07 19:15:05.982384 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:05.982358 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" Jul 07 19:15:06.051212 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:06.051183 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/430a211d-9ae5-43e6-9725-0bc75de82a9b-maas-api-tls\") pod \"430a211d-9ae5-43e6-9725-0bc75de82a9b\" (UID: \"430a211d-9ae5-43e6-9725-0bc75de82a9b\") " Jul 07 19:15:06.051428 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:06.051266 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-87674\" (UniqueName: \"kubernetes.io/projected/430a211d-9ae5-43e6-9725-0bc75de82a9b-kube-api-access-87674\") pod \"430a211d-9ae5-43e6-9725-0bc75de82a9b\" (UID: \"430a211d-9ae5-43e6-9725-0bc75de82a9b\") " Jul 07 19:15:06.051428 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:06.051295 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/430a211d-9ae5-43e6-9725-0bc75de82a9b-openshift-service-ca\") pod \"430a211d-9ae5-43e6-9725-0bc75de82a9b\" (UID: \"430a211d-9ae5-43e6-9725-0bc75de82a9b\") " Jul 07 19:15:06.051775 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:06.051748 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/430a211d-9ae5-43e6-9725-0bc75de82a9b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "430a211d-9ae5-43e6-9725-0bc75de82a9b" (UID: "430a211d-9ae5-43e6-9725-0bc75de82a9b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 19:15:06.053421 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:06.053352 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/430a211d-9ae5-43e6-9725-0bc75de82a9b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "430a211d-9ae5-43e6-9725-0bc75de82a9b" (UID: "430a211d-9ae5-43e6-9725-0bc75de82a9b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:15:06.053557 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:06.053534 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/430a211d-9ae5-43e6-9725-0bc75de82a9b-kube-api-access-87674" (OuterVolumeSpecName: "kube-api-access-87674") pod "430a211d-9ae5-43e6-9725-0bc75de82a9b" (UID: "430a211d-9ae5-43e6-9725-0bc75de82a9b"). InnerVolumeSpecName "kube-api-access-87674". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:15:06.152523 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:06.152471 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-87674\" (UniqueName: \"kubernetes.io/projected/430a211d-9ae5-43e6-9725-0bc75de82a9b-kube-api-access-87674\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 19:15:06.152523 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:06.152510 2564 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/430a211d-9ae5-43e6-9725-0bc75de82a9b-openshift-service-ca\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 19:15:06.152523 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:06.152525 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/430a211d-9ae5-43e6-9725-0bc75de82a9b-maas-api-tls\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 19:15:06.641256 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:06.641225 2564 generic.go:356] "Generic (PLEG): container finished" podID="430a211d-9ae5-43e6-9725-0bc75de82a9b" containerID="95c5c2fe11a9959483bdd017e6f07ffc50a128967382058df256578aec56a3f5" exitCode=0 Jul 07 19:15:06.641831 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:06.641267 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" Jul 07 19:15:06.641831 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:06.641273 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" event={"ID":"430a211d-9ae5-43e6-9725-0bc75de82a9b","Type":"ContainerDied","Data":"95c5c2fe11a9959483bdd017e6f07ffc50a128967382058df256578aec56a3f5"} Jul 07 19:15:06.641831 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:06.641371 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl" event={"ID":"430a211d-9ae5-43e6-9725-0bc75de82a9b","Type":"ContainerDied","Data":"d964a8af6786fc1b8a6585775a94444cc3ea4766f9e5acf558b50b51eb840fb5"} Jul 07 19:15:06.641831 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:06.641393 2564 scope.go:117] "RemoveContainer" containerID="95c5c2fe11a9959483bdd017e6f07ffc50a128967382058df256578aec56a3f5" Jul 07 19:15:06.651902 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:06.651885 2564 scope.go:117] "RemoveContainer" containerID="95c5c2fe11a9959483bdd017e6f07ffc50a128967382058df256578aec56a3f5" Jul 07 19:15:06.652163 ip-10-0-142-66 kubenswrapper[2564]: E0707 19:15:06.652144 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95c5c2fe11a9959483bdd017e6f07ffc50a128967382058df256578aec56a3f5\": container with ID starting with 95c5c2fe11a9959483bdd017e6f07ffc50a128967382058df256578aec56a3f5 not found: ID does not exist" containerID="95c5c2fe11a9959483bdd017e6f07ffc50a128967382058df256578aec56a3f5" Jul 07 19:15:06.652226 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:06.652171 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95c5c2fe11a9959483bdd017e6f07ffc50a128967382058df256578aec56a3f5"} err="failed to get container status \"95c5c2fe11a9959483bdd017e6f07ffc50a128967382058df256578aec56a3f5\": rpc error: code = NotFound desc = could not find container \"95c5c2fe11a9959483bdd017e6f07ffc50a128967382058df256578aec56a3f5\": container with ID starting with 95c5c2fe11a9959483bdd017e6f07ffc50a128967382058df256578aec56a3f5 not found: ID does not exist" Jul 07 19:15:06.667150 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:06.667123 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl"] Jul 07 19:15:06.672738 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:06.672711 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-914b2894-5b6b57fbb5-dgghl"] Jul 07 19:15:07.104299 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:15:07.104267 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="430a211d-9ae5-43e6-9725-0bc75de82a9b" path="/var/lib/kubelet/pods/430a211d-9ae5-43e6-9725-0bc75de82a9b/volumes" Jul 07 19:19:53.214780 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:19:53.214749 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/1.log" Jul 07 19:19:53.224802 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:19:53.224776 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/1.log" Jul 07 19:24:53.251841 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:24:53.251815 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/1.log" Jul 07 19:24:53.270684 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:24:53.270659 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/1.log" Jul 07 19:26:11.239475 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.239440 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd"] Jul 07 19:26:11.241859 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.239994 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="430a211d-9ae5-43e6-9725-0bc75de82a9b" containerName="maas-api" Jul 07 19:26:11.241859 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.240009 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="430a211d-9ae5-43e6-9725-0bc75de82a9b" containerName="maas-api" Jul 07 19:26:11.241859 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.240075 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="430a211d-9ae5-43e6-9725-0bc75de82a9b" containerName="maas-api" Jul 07 19:26:11.243131 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.243115 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:11.246480 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.246462 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-912fa0\"/\"openshift-service-ca.crt\"" Jul 07 19:26:11.246984 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.246964 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-912fa0\"/\"kube-root-ca.crt\"" Jul 07 19:26:11.247072 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.246992 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-912fa0\"/\"default-dockercfg-hwmrd\"" Jul 07 19:26:11.247072 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.247002 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-912fa0\"/\"sub-test-model-912fa0-kserve-self-signed-certs\"" Jul 07 19:26:11.256597 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.256561 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd"] Jul 07 19:26:11.337115 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.337090 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-dshm\") pod \"sub-test-model-912fa0-kserve-5885cf58b5-8qjqd\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:11.337244 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.337122 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d56j7\" (UniqueName: \"kubernetes.io/projected/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-kube-api-access-d56j7\") pod \"sub-test-model-912fa0-kserve-5885cf58b5-8qjqd\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:11.337244 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.337151 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-model-cache\") pod \"sub-test-model-912fa0-kserve-5885cf58b5-8qjqd\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:11.337244 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.337178 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-tmp-dir\") pod \"sub-test-model-912fa0-kserve-5885cf58b5-8qjqd\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:11.337244 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.337226 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-tls-certs\") pod \"sub-test-model-912fa0-kserve-5885cf58b5-8qjqd\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:11.337428 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.337279 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-home\") pod \"sub-test-model-912fa0-kserve-5885cf58b5-8qjqd\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:11.438224 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.438202 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-home\") pod \"sub-test-model-912fa0-kserve-5885cf58b5-8qjqd\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:11.438332 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.438263 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-dshm\") pod \"sub-test-model-912fa0-kserve-5885cf58b5-8qjqd\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:11.438332 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.438282 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d56j7\" (UniqueName: \"kubernetes.io/projected/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-kube-api-access-d56j7\") pod \"sub-test-model-912fa0-kserve-5885cf58b5-8qjqd\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:11.438332 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.438310 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-model-cache\") pod \"sub-test-model-912fa0-kserve-5885cf58b5-8qjqd\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:11.438471 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.438334 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-tmp-dir\") pod \"sub-test-model-912fa0-kserve-5885cf58b5-8qjqd\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:11.438471 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.438351 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-tls-certs\") pod \"sub-test-model-912fa0-kserve-5885cf58b5-8qjqd\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:11.438653 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.438629 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-home\") pod \"sub-test-model-912fa0-kserve-5885cf58b5-8qjqd\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:11.438729 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.438681 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-model-cache\") pod \"sub-test-model-912fa0-kserve-5885cf58b5-8qjqd\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:11.438786 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.438754 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-tmp-dir\") pod \"sub-test-model-912fa0-kserve-5885cf58b5-8qjqd\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:11.440535 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.440506 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-dshm\") pod \"sub-test-model-912fa0-kserve-5885cf58b5-8qjqd\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:11.440803 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.440788 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-tls-certs\") pod \"sub-test-model-912fa0-kserve-5885cf58b5-8qjqd\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:11.450359 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.450341 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d56j7\" (UniqueName: \"kubernetes.io/projected/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-kube-api-access-d56j7\") pod \"sub-test-model-912fa0-kserve-5885cf58b5-8qjqd\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:11.553099 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.553077 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:11.914760 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:11.914735 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd"] Jul 07 19:26:11.915565 ip-10-0-142-66 kubenswrapper[2564]: W0707 19:26:11.915542 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd0a0be1_9fd9_477b_b4c4_1038288d96d2.slice/crio-3b6ac11e42ebf4623abcf6d6036ae070c0788313b6362d9b318b18880604ac32 WatchSource:0}: Error finding container 3b6ac11e42ebf4623abcf6d6036ae070c0788313b6362d9b318b18880604ac32: Status 404 returned error can't find the container with id 3b6ac11e42ebf4623abcf6d6036ae070c0788313b6362d9b318b18880604ac32 Jul 07 19:26:12.477809 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:12.477777 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" event={"ID":"bd0a0be1-9fd9-477b-b4c4-1038288d96d2","Type":"ContainerStarted","Data":"60f025b365681497048f3b4a2c3abf9e7eff5fbfe44773ef7f7c64e2cb924947"} Jul 07 19:26:12.478237 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:12.477823 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" event={"ID":"bd0a0be1-9fd9-477b-b4c4-1038288d96d2","Type":"ContainerStarted","Data":"3b6ac11e42ebf4623abcf6d6036ae070c0788313b6362d9b318b18880604ac32"} Jul 07 19:26:12.497426 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:12.497374 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" podStartSLOduration=1.4973593539999999 podStartE2EDuration="1.497359354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:26:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:26:12.495923639 +0000 UTC m=+2179.915283873" watchObservedRunningTime="2026-07-07 19:26:12.497359354 +0000 UTC m=+2179.916719577" Jul 07 19:26:21.553483 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:21.553449 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:21.553899 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:21.553498 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:21.565943 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:21.565919 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:22.549336 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:22.549308 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:26:47.072466 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:26:47.072431 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd"] Jul 07 19:27:02.094344 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.094304 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" podUID="bd0a0be1-9fd9-477b-b4c4-1038288d96d2" containerName="main" containerID="cri-o://60f025b365681497048f3b4a2c3abf9e7eff5fbfe44773ef7f7c64e2cb924947" gracePeriod=45 Jul 07 19:27:02.483083 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.483056 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:27:02.643740 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.643708 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d56j7\" (UniqueName: \"kubernetes.io/projected/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-kube-api-access-d56j7\") pod \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " Jul 07 19:27:02.643877 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.643753 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-tmp-dir\") pod \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " Jul 07 19:27:02.643877 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.643801 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-dshm\") pod \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " Jul 07 19:27:02.643877 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.643840 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-tls-certs\") pod \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " Jul 07 19:27:02.644044 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.643884 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-model-cache\") pod \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " Jul 07 19:27:02.644044 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.643911 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-home\") pod \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\" (UID: \"bd0a0be1-9fd9-477b-b4c4-1038288d96d2\") " Jul 07 19:27:02.644044 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.644028 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bd0a0be1-9fd9-477b-b4c4-1038288d96d2" (UID: "bd0a0be1-9fd9-477b-b4c4-1038288d96d2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:27:02.644200 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.644158 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-model-cache" (OuterVolumeSpecName: "model-cache") pod "bd0a0be1-9fd9-477b-b4c4-1038288d96d2" (UID: "bd0a0be1-9fd9-477b-b4c4-1038288d96d2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:27:02.644259 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.644232 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-home" (OuterVolumeSpecName: "home") pod "bd0a0be1-9fd9-477b-b4c4-1038288d96d2" (UID: "bd0a0be1-9fd9-477b-b4c4-1038288d96d2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:27:02.644311 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.644254 2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-tmp-dir\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 19:27:02.646277 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.646250 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-dshm" (OuterVolumeSpecName: "dshm") pod "bd0a0be1-9fd9-477b-b4c4-1038288d96d2" (UID: "bd0a0be1-9fd9-477b-b4c4-1038288d96d2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 19:27:02.646485 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.646464 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bd0a0be1-9fd9-477b-b4c4-1038288d96d2" (UID: "bd0a0be1-9fd9-477b-b4c4-1038288d96d2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 19:27:02.646545 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.646480 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-kube-api-access-d56j7" (OuterVolumeSpecName: "kube-api-access-d56j7") pod "bd0a0be1-9fd9-477b-b4c4-1038288d96d2" (UID: "bd0a0be1-9fd9-477b-b4c4-1038288d96d2"). InnerVolumeSpecName "kube-api-access-d56j7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 19:27:02.745123 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.745072 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d56j7\" (UniqueName: \"kubernetes.io/projected/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-kube-api-access-d56j7\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 19:27:02.745123 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.745092 2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-dshm\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 19:27:02.745123 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.745102 2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-tls-certs\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 19:27:02.745123 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.745110 2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-model-cache\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 19:27:02.745123 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.745117 2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd0a0be1-9fd9-477b-b4c4-1038288d96d2-home\") on node \"ip-10-0-142-66.ec2.internal\" DevicePath \"\"" Jul 07 19:27:02.787123 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.787097 2564 generic.go:356] "Generic (PLEG): container finished" podID="bd0a0be1-9fd9-477b-b4c4-1038288d96d2" containerID="60f025b365681497048f3b4a2c3abf9e7eff5fbfe44773ef7f7c64e2cb924947" exitCode=0 Jul 07 19:27:02.787244 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.787147 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" Jul 07 19:27:02.787244 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.787176 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" event={"ID":"bd0a0be1-9fd9-477b-b4c4-1038288d96d2","Type":"ContainerDied","Data":"60f025b365681497048f3b4a2c3abf9e7eff5fbfe44773ef7f7c64e2cb924947"} Jul 07 19:27:02.787244 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.787219 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd" event={"ID":"bd0a0be1-9fd9-477b-b4c4-1038288d96d2","Type":"ContainerDied","Data":"3b6ac11e42ebf4623abcf6d6036ae070c0788313b6362d9b318b18880604ac32"} Jul 07 19:27:02.787244 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.787240 2564 scope.go:117] "RemoveContainer" containerID="60f025b365681497048f3b4a2c3abf9e7eff5fbfe44773ef7f7c64e2cb924947" Jul 07 19:27:02.796535 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.796520 2564 scope.go:117] "RemoveContainer" containerID="60f025b365681497048f3b4a2c3abf9e7eff5fbfe44773ef7f7c64e2cb924947" Jul 07 19:27:02.796812 ip-10-0-142-66 kubenswrapper[2564]: E0707 19:27:02.796791 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60f025b365681497048f3b4a2c3abf9e7eff5fbfe44773ef7f7c64e2cb924947\": container with ID starting with 60f025b365681497048f3b4a2c3abf9e7eff5fbfe44773ef7f7c64e2cb924947 not found: ID does not exist" containerID="60f025b365681497048f3b4a2c3abf9e7eff5fbfe44773ef7f7c64e2cb924947" Jul 07 19:27:02.796859 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.796822 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60f025b365681497048f3b4a2c3abf9e7eff5fbfe44773ef7f7c64e2cb924947"} err="failed to get container status \"60f025b365681497048f3b4a2c3abf9e7eff5fbfe44773ef7f7c64e2cb924947\": rpc error: code = NotFound desc = could not find container \"60f025b365681497048f3b4a2c3abf9e7eff5fbfe44773ef7f7c64e2cb924947\": container with ID starting with 60f025b365681497048f3b4a2c3abf9e7eff5fbfe44773ef7f7c64e2cb924947 not found: ID does not exist" Jul 07 19:27:02.808863 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.808841 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd"] Jul 07 19:27:02.815603 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:02.815556 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-912fa0/sub-test-model-912fa0-kserve-5885cf58b5-8qjqd"] Jul 07 19:27:03.112487 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:27:03.112453 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd0a0be1-9fd9-477b-b4c4-1038288d96d2" path="/var/lib/kubelet/pods/bd0a0be1-9fd9-477b-b4c4-1038288d96d2/volumes" Jul 07 19:29:16.587625 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:16.587539 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-lnm7t_8bb75d2e-60c8-4969-ba8d-abfa2cb2a0d1/manager/0.log" Jul 07 19:29:16.711187 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:16.711160 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-9ldv7_3541e00e-0398-42b2-94f3-805115d3f75d/manager/0.log" Jul 07 19:29:16.835796 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:16.835771 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6d85fdd848-krbqg_218f5aaa-287e-4243-ba90-a62a1912b472/maas-api/0.log" Jul 07 19:29:17.226109 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:17.226075 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d8f857c75-9w7hs_074b8f0e-6b1b-4c33-a595-75f1fa7d3da3/manager/0.log" Jul 07 19:29:17.339076 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:17.339048 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-kx4hr_01d9c78d-f139-40a5-beb1-544649d9794c/server/0.log" Jul 07 19:29:17.450831 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:17.450809 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-ttzl8_9ec43cc0-930a-4620-b06d-6d4d6a2d4e33/manager/0.log" Jul 07 19:29:17.810810 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:17.810785 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64dd56d745-xhpqv_c262c19a-4160-4238-927d-1f282a06b415/copy-manifests/0.log" Jul 07 19:29:17.817855 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:17.817834 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64dd56d745-xhpqv_c262c19a-4160-4238-927d-1f282a06b415/manager/0.log" Jul 07 19:29:17.926876 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:17.926858 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-649zj_122d33f0-a664-4055-9ded-4d2f5dcc4331/postgres/0.log" Jul 07 19:29:18.698175 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:18.698148 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd_65401510-654a-4e3c-b89b-809ee03e398c/util/0.log" Jul 07 19:29:18.716497 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:18.716475 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd_65401510-654a-4e3c-b89b-809ee03e398c/pull/0.log" Jul 07 19:29:18.736068 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:18.736048 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd_65401510-654a-4e3c-b89b-809ee03e398c/extract/0.log" Jul 07 19:29:18.852849 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:18.852833 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2_67ee7edd-7a6b-46a6-869e-d2054008e73c/util/0.log" Jul 07 19:29:18.860633 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:18.860611 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2_67ee7edd-7a6b-46a6-869e-d2054008e73c/pull/0.log" Jul 07 19:29:18.867614 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:18.867589 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2_67ee7edd-7a6b-46a6-869e-d2054008e73c/extract/0.log" Jul 07 19:29:18.974639 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:18.974568 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h_a0969383-1c72-470a-a990-7ef7d6a752d2/util/0.log" Jul 07 19:29:18.982430 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:18.982414 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h_a0969383-1c72-470a-a990-7ef7d6a752d2/pull/0.log" Jul 07 19:29:18.997801 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:18.997783 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h_a0969383-1c72-470a-a990-7ef7d6a752d2/extract/0.log" Jul 07 19:29:19.110655 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:19.110630 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl_5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf/util/0.log" Jul 07 19:29:19.123746 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:19.123720 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl_5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf/pull/0.log" Jul 07 19:29:19.135004 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:19.134984 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl_5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf/extract/0.log" Jul 07 19:29:19.403718 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:19.403694 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-pxmnc_e2c66c7f-f736-4cee-a590-9a3c64686947/manager/0.log" Jul 07 19:29:19.513960 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:19.513941 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4wnh5_46db6b91-a179-4f25-bd7b-7a00183e071c/manager/0.log" Jul 07 19:29:19.620520 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:19.620499 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-268dw_3a19cc62-b466-4cd7-8cba-4f573f4c3035/kuadrant-console-plugin/0.log" Jul 07 19:29:19.735532 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:19.735481 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-drrrt_57988adf-5c24-43ce-8c48-9fa1d0a66f43/registry-server/0.log" Jul 07 19:29:19.853168 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:19.853146 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-5sggt_07ab7460-9b7c-4304-bfc0-5b9d7c704746/manager/0.log" Jul 07 19:29:20.085941 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:20.085921 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-swf96_12399cf6-ace4-452a-80e0-c74a24c16a41/manager/0.log" Jul 07 19:29:20.782136 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:20.782113 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5598bb98cc-kjl55_71c43c90-08d3-4214-96ca-2b87e14e1df8/kube-auth-proxy/0.log" Jul 07 19:29:21.014175 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:21.014155 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-474mg_c0fba180-a064-4905-86b3-df268d3d51c8/payload-pre-processing/0.log" Jul 07 19:29:21.132230 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:21.132169 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-wzdjn_13d7fd32-6207-4178-ac83-914df4da7e9f/payload-processing/0.log" Jul 07 19:29:21.248862 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:21.248843 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7dd847dbd5-mq8mz_2003b9f1-daee-437f-8f44-9ce42ae8f6bd/router/0.log" Jul 07 19:29:21.698431 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:21.698408 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-tttn5_53e2af7b-73b5-4ca5-8e4a-7efcbe8ebf5e/main/0.log" Jul 07 19:29:21.923060 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:21.923037 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db6bggp_315c54b5-0d92-4893-b3e4-c08f079befd3/main/0.log" Jul 07 19:29:22.031749 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:22.031730 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-6hwjk_3a9eb5d0-823f-43d8-8e5e-5f74651dc753/main/0.log" Jul 07 19:29:29.390756 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:29.390728 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hslvd_2e3ec6d8-dbb1-4610-b924-9d3fec6db0bf/global-pull-secret-syncer/0.log" Jul 07 19:29:29.571626 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:29.571568 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vw7p6_4b1d9d37-d1f8-4ebd-b96d-0a11e50a90e4/konnectivity-agent/0.log" Jul 07 19:29:29.652243 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:29.652182 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-66.ec2.internal_63225eaf1c1ca60865989b36d98a70bf/haproxy/0.log" Jul 07 19:29:33.371324 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:33.371298 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd_65401510-654a-4e3c-b89b-809ee03e398c/extract/0.log" Jul 07 19:29:33.436137 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:33.436072 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd_65401510-654a-4e3c-b89b-809ee03e398c/util/0.log" Jul 07 19:29:33.461191 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:33.461172 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759smtdd_65401510-654a-4e3c-b89b-809ee03e398c/pull/0.log" Jul 07 19:29:33.497456 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:33.497436 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2_67ee7edd-7a6b-46a6-869e-d2054008e73c/extract/0.log" Jul 07 19:29:33.525938 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:33.525922 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2_67ee7edd-7a6b-46a6-869e-d2054008e73c/util/0.log" Jul 07 19:29:33.549756 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:33.549736 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0v4xc2_67ee7edd-7a6b-46a6-869e-d2054008e73c/pull/0.log" Jul 07 19:29:33.593453 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:33.593433 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h_a0969383-1c72-470a-a990-7ef7d6a752d2/extract/0.log" Jul 07 19:29:33.618169 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:33.618146 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h_a0969383-1c72-470a-a990-7ef7d6a752d2/util/0.log" Jul 07 19:29:33.643209 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:33.643184 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qfd5h_a0969383-1c72-470a-a990-7ef7d6a752d2/pull/0.log" Jul 07 19:29:33.673507 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:33.673490 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl_5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf/extract/0.log" Jul 07 19:29:33.709428 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:33.709375 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl_5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf/util/0.log" Jul 07 19:29:33.731052 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:33.731032 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tj5gl_5ad39d3b-c5ce-439e-9fca-4eeaf48a3fbf/pull/0.log" Jul 07 19:29:34.279908 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:34.279887 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-pxmnc_e2c66c7f-f736-4cee-a590-9a3c64686947/manager/0.log" Jul 07 19:29:34.321623 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:34.321605 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-4wnh5_46db6b91-a179-4f25-bd7b-7a00183e071c/manager/0.log" Jul 07 19:29:34.366132 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:34.366113 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-268dw_3a19cc62-b466-4cd7-8cba-4f573f4c3035/kuadrant-console-plugin/0.log" Jul 07 19:29:34.426587 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:34.426551 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-drrrt_57988adf-5c24-43ce-8c48-9fa1d0a66f43/registry-server/0.log" Jul 07 19:29:34.544511 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:34.544447 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-5sggt_07ab7460-9b7c-4304-bfc0-5b9d7c704746/manager/0.log" Jul 07 19:29:34.655999 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:34.655970 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-swf96_12399cf6-ace4-452a-80e0-c74a24c16a41/manager/0.log" Jul 07 19:29:36.353083 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:36.353058 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-b5nnl_10350357-c651-4b89-8497-7e16a1b6c701/cluster-monitoring-operator/0.log" Jul 07 19:29:36.757392 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:36.757368 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r6ghq_920236cd-e793-4a32-bce6-ddb1847993bf/node-exporter/0.log" Jul 07 19:29:36.782314 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:36.782289 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r6ghq_920236cd-e793-4a32-bce6-ddb1847993bf/kube-rbac-proxy/0.log" Jul 07 19:29:36.806055 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:36.806036 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r6ghq_920236cd-e793-4a32-bce6-ddb1847993bf/init-textfile/0.log" Jul 07 19:29:37.118912 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:37.118890 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-4w8fq_fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f/prometheus-operator/0.log" Jul 07 19:29:37.139490 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:37.139469 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-4w8fq_fb3ae3d6-9b9b-4f4f-900d-81b18e0d854f/kube-rbac-proxy/0.log" Jul 07 19:29:37.166156 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:37.166140 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-9gts2_5c7e6133-de5f-4d57-93dd-d2f45497e09e/prometheus-operator-admission-webhook/0.log" Jul 07 19:29:39.317624 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.317556 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/1.log" Jul 07 19:29:39.321856 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.321835 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/2.log" Jul 07 19:29:39.405998 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.405977 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v"] Jul 07 19:29:39.406490 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.406468 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd0a0be1-9fd9-477b-b4c4-1038288d96d2" containerName="main" Jul 07 19:29:39.406490 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.406488 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd0a0be1-9fd9-477b-b4c4-1038288d96d2" containerName="main" Jul 07 19:29:39.406665 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.406634 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd0a0be1-9fd9-477b-b4c4-1038288d96d2" containerName="main" Jul 07 19:29:39.409859 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.409842 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" Jul 07 19:29:39.412208 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.412190 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5qrll\"/\"kube-root-ca.crt\"" Jul 07 19:29:39.412297 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.412190 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5qrll\"/\"default-dockercfg-d64b8\"" Jul 07 19:29:39.412915 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.412899 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5qrll\"/\"openshift-service-ca.crt\"" Jul 07 19:29:39.420240 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.420214 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v"] Jul 07 19:29:39.478425 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.478402 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1d5e29a-60c9-46d5-95ac-17fc58d04375-sys\") pod \"perf-node-gather-daemonset-hmb8v\" (UID: \"f1d5e29a-60c9-46d5-95ac-17fc58d04375\") " pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" Jul 07 19:29:39.478522 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.478445 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccvkc\" (UniqueName: \"kubernetes.io/projected/f1d5e29a-60c9-46d5-95ac-17fc58d04375-kube-api-access-ccvkc\") pod \"perf-node-gather-daemonset-hmb8v\" (UID: \"f1d5e29a-60c9-46d5-95ac-17fc58d04375\") " pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" Jul 07 19:29:39.478563 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.478524 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f1d5e29a-60c9-46d5-95ac-17fc58d04375-lib-modules\") pod \"perf-node-gather-daemonset-hmb8v\" (UID: \"f1d5e29a-60c9-46d5-95ac-17fc58d04375\") " pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" Jul 07 19:29:39.478563 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.478551 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f1d5e29a-60c9-46d5-95ac-17fc58d04375-proc\") pod \"perf-node-gather-daemonset-hmb8v\" (UID: \"f1d5e29a-60c9-46d5-95ac-17fc58d04375\") " pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" Jul 07 19:29:39.478655 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.478595 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f1d5e29a-60c9-46d5-95ac-17fc58d04375-podres\") pod \"perf-node-gather-daemonset-hmb8v\" (UID: \"f1d5e29a-60c9-46d5-95ac-17fc58d04375\") " pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" Jul 07 19:29:39.579688 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.579622 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f1d5e29a-60c9-46d5-95ac-17fc58d04375-podres\") pod \"perf-node-gather-daemonset-hmb8v\" (UID: \"f1d5e29a-60c9-46d5-95ac-17fc58d04375\") " pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" Jul 07 19:29:39.579688 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.579667 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1d5e29a-60c9-46d5-95ac-17fc58d04375-sys\") pod \"perf-node-gather-daemonset-hmb8v\" (UID: \"f1d5e29a-60c9-46d5-95ac-17fc58d04375\") " pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" Jul 07 19:29:39.579841 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.579700 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccvkc\" (UniqueName: \"kubernetes.io/projected/f1d5e29a-60c9-46d5-95ac-17fc58d04375-kube-api-access-ccvkc\") pod \"perf-node-gather-daemonset-hmb8v\" (UID: \"f1d5e29a-60c9-46d5-95ac-17fc58d04375\") " pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" Jul 07 19:29:39.579841 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.579739 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f1d5e29a-60c9-46d5-95ac-17fc58d04375-lib-modules\") pod \"perf-node-gather-daemonset-hmb8v\" (UID: \"f1d5e29a-60c9-46d5-95ac-17fc58d04375\") " pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" Jul 07 19:29:39.579841 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.579755 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f1d5e29a-60c9-46d5-95ac-17fc58d04375-proc\") pod \"perf-node-gather-daemonset-hmb8v\" (UID: \"f1d5e29a-60c9-46d5-95ac-17fc58d04375\") " pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" Jul 07 19:29:39.579841 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.579752 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1d5e29a-60c9-46d5-95ac-17fc58d04375-sys\") pod \"perf-node-gather-daemonset-hmb8v\" (UID: \"f1d5e29a-60c9-46d5-95ac-17fc58d04375\") " pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" Jul 07 19:29:39.579841 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.579810 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f1d5e29a-60c9-46d5-95ac-17fc58d04375-podres\") pod \"perf-node-gather-daemonset-hmb8v\" (UID: \"f1d5e29a-60c9-46d5-95ac-17fc58d04375\") " pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" Jul 07 19:29:39.579841 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.579828 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f1d5e29a-60c9-46d5-95ac-17fc58d04375-proc\") pod \"perf-node-gather-daemonset-hmb8v\" (UID: \"f1d5e29a-60c9-46d5-95ac-17fc58d04375\") " pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" Jul 07 19:29:39.580040 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.579882 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f1d5e29a-60c9-46d5-95ac-17fc58d04375-lib-modules\") pod \"perf-node-gather-daemonset-hmb8v\" (UID: \"f1d5e29a-60c9-46d5-95ac-17fc58d04375\") " pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" Jul 07 19:29:39.590656 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.590636 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccvkc\" (UniqueName: \"kubernetes.io/projected/f1d5e29a-60c9-46d5-95ac-17fc58d04375-kube-api-access-ccvkc\") pod \"perf-node-gather-daemonset-hmb8v\" (UID: \"f1d5e29a-60c9-46d5-95ac-17fc58d04375\") " pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" Jul 07 19:29:39.720388 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.720369 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" Jul 07 19:29:39.881478 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.881411 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7975cb9647-jdmtd_6aa77157-0983-4d33-9aa8-49fc8d996c5e/console/0.log" Jul 07 19:29:39.916016 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:39.915996 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-qdtn2_85c2ce25-fee8-43a2-b63c-4ce54bca5057/download-server/0.log" Jul 07 19:29:40.061132 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:40.061106 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v"] Jul 07 19:29:40.062065 ip-10-0-142-66 kubenswrapper[2564]: W0707 19:29:40.062041 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf1d5e29a_60c9_46d5_95ac_17fc58d04375.slice/crio-b5fde3ff9ec7cc766a028de31c29c330094a555cc99520b27cf3da1e196a8f0c WatchSource:0}: Error finding container b5fde3ff9ec7cc766a028de31c29c330094a555cc99520b27cf3da1e196a8f0c: Status 404 returned error can't find the container with id b5fde3ff9ec7cc766a028de31c29c330094a555cc99520b27cf3da1e196a8f0c Jul 07 19:29:40.548168 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:40.548133 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-55mgr_29bdac91-8043-4225-9c92-0d90a7d51003/volume-data-source-validator/0.log" Jul 07 19:29:40.689223 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:40.689196 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" event={"ID":"f1d5e29a-60c9-46d5-95ac-17fc58d04375","Type":"ContainerStarted","Data":"05897fdde3dfe9fbe63a4a41d5f0a3d03edd3bd711cd3ef3a5973a364c127d41"} Jul 07 19:29:40.689346 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:40.689244 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" event={"ID":"f1d5e29a-60c9-46d5-95ac-17fc58d04375","Type":"ContainerStarted","Data":"b5fde3ff9ec7cc766a028de31c29c330094a555cc99520b27cf3da1e196a8f0c"} Jul 07 19:29:40.689346 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:40.689269 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" Jul 07 19:29:40.708381 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:40.708340 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" podStartSLOduration=1.708326167 podStartE2EDuration="1.708326167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:29:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:29:40.706543113 +0000 UTC m=+2388.125903337" watchObservedRunningTime="2026-07-07 19:29:40.708326167 +0000 UTC m=+2388.127686390" Jul 07 19:29:41.701448 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:41.701424 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wvsd8_23e1f849-e113-4a42-bc58-448388e582ce/dns/0.log" Jul 07 19:29:41.732526 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:41.732482 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wvsd8_23e1f849-e113-4a42-bc58-448388e582ce/kube-rbac-proxy/0.log" Jul 07 19:29:41.847773 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:41.847756 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tmlg4_d305832c-a3ec-453e-9135-697b0b80c619/dns-node-resolver/0.log" Jul 07 19:29:42.485942 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:42.485905 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v6tvj_ac705cb6-fdca-4ffe-b46b-7e299b498b3e/node-ca/0.log" Jul 07 19:29:43.692257 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:43.692225 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5598bb98cc-kjl55_71c43c90-08d3-4214-96ca-2b87e14e1df8/kube-auth-proxy/0.log" Jul 07 19:29:43.802768 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:43.802744 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-474mg_c0fba180-a064-4905-86b3-df268d3d51c8/payload-pre-processing/0.log" Jul 07 19:29:43.836951 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:43.836927 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-wzdjn_13d7fd32-6207-4178-ac83-914df4da7e9f/payload-processing/0.log" Jul 07 19:29:43.868838 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:43.868804 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7dd847dbd5-mq8mz_2003b9f1-daee-437f-8f44-9ce42ae8f6bd/router/0.log" Jul 07 19:29:44.560732 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:44.560710 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9s59k_76ee52c5-2d6b-4c13-b50f-312157fc165f/serve-healthcheck-canary/0.log" Jul 07 19:29:45.066644 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:45.066600 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-qx6d9_c3b0a294-e7a3-49fa-a089-92d4bc757e8e/insights-operator/0.log" Jul 07 19:29:45.067051 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:45.066801 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-qx6d9_c3b0a294-e7a3-49fa-a089-92d4bc757e8e/insights-operator/1.log" Jul 07 19:29:45.091362 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:45.091323 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ps8ws_fb613ac7-eae7-4e21-92ae-66891f614319/kube-rbac-proxy/0.log" Jul 07 19:29:45.125109 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:45.125088 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ps8ws_fb613ac7-eae7-4e21-92ae-66891f614319/exporter/0.log" Jul 07 19:29:45.152980 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:45.152950 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ps8ws_fb613ac7-eae7-4e21-92ae-66891f614319/extractor/0.log" Jul 07 19:29:46.704146 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:46.704116 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5qrll/perf-node-gather-daemonset-hmb8v" Jul 07 19:29:47.689110 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:47.689081 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-lnm7t_8bb75d2e-60c8-4969-ba8d-abfa2cb2a0d1/manager/0.log" Jul 07 19:29:47.823319 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:47.823289 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-9ldv7_3541e00e-0398-42b2-94f3-805115d3f75d/manager/0.log" Jul 07 19:29:47.861284 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:47.861261 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6d85fdd848-krbqg_218f5aaa-287e-4243-ba90-a62a1912b472/maas-api/0.log" Jul 07 19:29:48.012066 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:48.012043 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d8f857c75-9w7hs_074b8f0e-6b1b-4c33-a595-75f1fa7d3da3/manager/0.log" Jul 07 19:29:48.042391 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:48.042370 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-kx4hr_01d9c78d-f139-40a5-beb1-544649d9794c/server/0.log" Jul 07 19:29:48.118221 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:48.118191 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-ttzl8_9ec43cc0-930a-4620-b06d-6d4d6a2d4e33/manager/0.log" Jul 07 19:29:48.375111 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:48.375035 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64dd56d745-xhpqv_c262c19a-4160-4238-927d-1f282a06b415/manager/0.log" Jul 07 19:29:48.400483 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:48.400465 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-64dd56d745-xhpqv_c262c19a-4160-4238-927d-1f282a06b415/copy-manifests/0.log" Jul 07 19:29:48.430504 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:48.430486 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-649zj_122d33f0-a664-4055-9ded-4d2f5dcc4331/postgres/0.log" Jul 07 19:29:49.627568 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:49.627544 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5598bb98cc-kjl55_71c43c90-08d3-4214-96ca-2b87e14e1df8/kube-auth-proxy/0.log" Jul 07 19:29:49.761248 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:49.761224 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-474mg_c0fba180-a064-4905-86b3-df268d3d51c8/payload-pre-processing/0.log" Jul 07 19:29:49.797950 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:49.797927 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-wzdjn_13d7fd32-6207-4178-ac83-914df4da7e9f/payload-processing/0.log" Jul 07 19:29:49.823978 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:49.823961 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7dd847dbd5-mq8mz_2003b9f1-daee-437f-8f44-9ce42ae8f6bd/router/0.log" Jul 07 19:29:49.882339 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:49.882320 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-rl77f_738a31b4-ff84-4b3d-b282-fc47a5ac647f/openshift-lws-operator/0.log" Jul 07 19:29:51.215601 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:51.215561 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5598bb98cc-kjl55_71c43c90-08d3-4214-96ca-2b87e14e1df8/kube-auth-proxy/0.log" Jul 07 19:29:51.291638 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:51.291603 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-474mg_c0fba180-a064-4905-86b3-df268d3d51c8/payload-pre-processing/0.log" Jul 07 19:29:51.314675 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:51.314610 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-wzdjn_13d7fd32-6207-4178-ac83-914df4da7e9f/payload-processing/0.log" Jul 07 19:29:51.331782 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:51.331764 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7dd847dbd5-mq8mz_2003b9f1-daee-437f-8f44-9ce42ae8f6bd/router/0.log" Jul 07 19:29:53.297373 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:53.297346 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/1.log" Jul 07 19:29:53.309181 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:53.309163 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-vcwql_7a3286ca-3966-48e7-a4d9-9830a7fb1063/console-operator/1.log" Jul 07 19:29:55.201217 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:55.201167 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-blnhp_1ce43d53-3718-49e6-a60e-ce5ca36877ca/kube-storage-version-migrator-operator/1.log" Jul 07 19:29:55.201979 ip-10-0-142-66 kubenswrapper[2564]: I0707 19:29:55.201963 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-blnhp_1ce43d53-3718-49e6-a60e-ce5ca36877ca/kube-storage-version-migrator-operator/0.log"