Jul 07 18:49:46.170678 ip-10-0-135-81 systemd[1]: Starting Kubernetes Kubelet... Jul 07 18:49:46.519863 ip-10-0-135-81 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 18:49:46.519863 ip-10-0-135-81 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 18:49:46.519863 ip-10-0-135-81 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 18:49:46.519863 ip-10-0-135-81 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 18:49:46.519863 ip-10-0-135-81 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 18:49:46.519863 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.480534 2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 18:49:46.519863 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483740 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:49:46.519863 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483752 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:49:46.519863 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483755 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:49:46.519863 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483758 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:49:46.519863 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483762 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:49:46.519863 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483764 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:49:46.519863 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483767 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:49:46.519863 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483770 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:49:46.522910 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483773 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:49:46.522910 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483775 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:49:46.522910 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483778 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:49:46.522910 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483781 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:49:46.522910 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483783 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:49:46.522910 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483786 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:49:46.522910 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483788 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:49:46.522910 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483791 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:49:46.522910 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483793 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:49:46.522910 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483796 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:49:46.522910 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483798 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:49:46.522910 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483801 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:49:46.522910 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483804 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:49:46.522910 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483806 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:49:46.522910 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483813 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:49:46.522910 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483815 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:49:46.522910 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483818 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:49:46.522910 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483820 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:49:46.522910 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483824 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:49:46.522910 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483829 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:49:46.523715 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483832 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:49:46.523715 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483835 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:49:46.523715 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483839 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:49:46.523715 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483842 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:49:46.523715 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483845 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:49:46.523715 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483847 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:49:46.523715 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483850 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:49:46.523715 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483852 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:49:46.523715 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483855 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:49:46.523715 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483857 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:49:46.523715 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483860 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:49:46.523715 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483862 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:49:46.523715 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483865 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:49:46.523715 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483867 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:49:46.523715 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483870 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:49:46.523715 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483873 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:49:46.523715 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483875 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:49:46.523715 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483877 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:49:46.523715 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483880 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:49:46.523715 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483882 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:49:46.526458 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483885 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:49:46.526458 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483887 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:49:46.526458 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483889 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:49:46.526458 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483892 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:49:46.526458 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483894 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:49:46.526458 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483897 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:49:46.526458 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483899 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:49:46.526458 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483901 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:49:46.526458 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483904 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:49:46.526458 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483906 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:49:46.526458 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483909 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:49:46.526458 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483911 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:49:46.526458 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483914 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:49:46.526458 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483916 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:49:46.526458 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483919 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:49:46.526458 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483921 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:49:46.526458 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483924 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:49:46.526458 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483927 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:49:46.526458 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483930 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:49:46.527214 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483932 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:49:46.527214 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483934 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:49:46.527214 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483937 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:49:46.527214 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483939 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:49:46.527214 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483942 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:49:46.527214 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483945 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:49:46.527214 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483947 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:49:46.527214 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483951 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:49:46.527214 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483954 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:49:46.527214 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483956 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:49:46.527214 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483958 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:49:46.527214 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483961 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:49:46.527214 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483963 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:49:46.527214 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483966 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:49:46.527214 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483968 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:49:46.527214 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483971 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:49:46.527214 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483973 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:49:46.527214 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483975 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:49:46.527214 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483977 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:49:46.527214 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483980 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:49:46.531116 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483982 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:49:46.531116 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483985 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:49:46.531116 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483987 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:49:46.531116 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483990 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:49:46.531116 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483993 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:49:46.531116 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483995 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:49:46.531116 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.483998 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:49:46.531116 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484001 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:49:46.531116 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484003 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:49:46.531116 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484007 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:49:46.531116 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484010 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:49:46.531116 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484013 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:49:46.531116 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484687 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:49:46.531116 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484694 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:49:46.531116 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484698 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:49:46.531116 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484700 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:49:46.531116 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484703 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:49:46.531116 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484706 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:49:46.531116 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484709 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:49:46.531116 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484711 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:49:46.531890 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484714 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:49:46.531890 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484717 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:49:46.531890 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484719 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:49:46.531890 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484722 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:49:46.531890 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484725 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:49:46.531890 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484727 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:49:46.531890 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484730 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:49:46.531890 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484733 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:49:46.531890 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484735 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:49:46.531890 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484738 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:49:46.531890 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484740 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:49:46.531890 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484742 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:49:46.531890 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484746 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:49:46.531890 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484750 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:49:46.531890 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484753 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:49:46.531890 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484756 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:49:46.531890 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484760 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:49:46.531890 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484762 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:49:46.531890 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484765 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:49:46.536619 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484768 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:49:46.536619 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484770 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:49:46.536619 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484773 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:49:46.536619 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484775 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:49:46.536619 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484778 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:49:46.536619 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484782 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:49:46.536619 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484784 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:49:46.536619 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484786 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:49:46.536619 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484789 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:49:46.536619 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484791 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:49:46.536619 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484794 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:49:46.536619 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484797 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:49:46.536619 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484799 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:49:46.536619 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484801 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:49:46.536619 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484804 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:49:46.536619 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484806 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:49:46.536619 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484809 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:49:46.536619 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484811 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:49:46.536619 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484815 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:49:46.536619 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484818 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:49:46.537454 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484821 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:49:46.537454 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484824 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:49:46.537454 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484828 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:49:46.537454 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484831 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:49:46.537454 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484834 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:49:46.537454 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484836 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:49:46.537454 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484839 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:49:46.537454 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484842 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:49:46.537454 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484844 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:49:46.537454 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484847 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:49:46.537454 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484849 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:49:46.537454 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484852 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:49:46.537454 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484854 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:49:46.537454 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484856 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:49:46.537454 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484860 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:49:46.537454 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484864 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:49:46.537454 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484867 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:49:46.537454 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484869 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:49:46.537454 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484871 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:49:46.538318 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484874 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:49:46.538318 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484876 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:49:46.538318 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484879 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:49:46.538318 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484881 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:49:46.538318 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484884 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:49:46.538318 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484886 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:49:46.538318 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484889 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:49:46.538318 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484891 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:49:46.538318 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484893 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:49:46.538318 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484896 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:49:46.538318 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484898 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:49:46.538318 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484901 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:49:46.538318 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484903 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:49:46.538318 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484906 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:49:46.538318 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484909 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:49:46.538318 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484911 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:49:46.538318 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484913 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:49:46.538318 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484916 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:49:46.538318 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484918 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:49:46.538318 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484920 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:49:46.541666 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484923 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:49:46.541666 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484925 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:49:46.541666 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484928 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:49:46.541666 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484930 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:49:46.541666 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484932 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:49:46.541666 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484935 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:49:46.541666 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484937 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:49:46.541666 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484939 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:49:46.541666 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484941 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:49:46.541666 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484944 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:49:46.541666 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484946 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:49:46.541666 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484948 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:49:46.541666 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.484950 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:49:46.541666 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485293 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:49:46.541666 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485298 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:49:46.541666 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485300 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:49:46.541666 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485303 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:49:46.541666 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485306 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:49:46.541666 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485308 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:49:46.541666 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485311 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:49:46.542280 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485314 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:49:46.542280 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485316 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:49:46.542280 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485319 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:49:46.542280 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485322 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:49:46.542280 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485326 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:49:46.542280 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485329 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:49:46.542280 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485331 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:49:46.542280 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485334 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:49:46.542280 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485337 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:49:46.542280 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485339 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:49:46.542280 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485341 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:49:46.542280 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485344 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:49:46.542280 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485346 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:49:46.542280 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485349 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:49:46.542280 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485351 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:49:46.542280 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485354 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:49:46.542280 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485356 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:49:46.542280 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485359 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:49:46.542280 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485361 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:49:46.542280 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485364 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:49:46.543094 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485367 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:49:46.543094 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485369 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:49:46.543094 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485371 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:49:46.543094 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485373 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:49:46.543094 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485376 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:49:46.543094 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485378 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:49:46.543094 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485383 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:49:46.543094 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485386 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:49:46.543094 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485388 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:49:46.543094 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485409 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:49:46.543094 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485413 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:49:46.543094 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485415 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:49:46.543094 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485419 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:49:46.543094 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485422 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:49:46.543094 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485426 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:49:46.543094 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485429 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:49:46.543094 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485432 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:49:46.543094 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485434 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:49:46.543094 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485437 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:49:46.545342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485440 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:49:46.545342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485442 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:49:46.545342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485445 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:49:46.545342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485448 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:49:46.545342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485450 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:49:46.545342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485452 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:49:46.545342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485455 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:49:46.545342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485457 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:49:46.545342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485460 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:49:46.545342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485462 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:49:46.545342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485466 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:49:46.545342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485469 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:49:46.545342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485473 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:49:46.545342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485475 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:49:46.545342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485478 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:49:46.545342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485481 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:49:46.545342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485483 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:49:46.545342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485488 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:49:46.545342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485491 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:49:46.545342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485495 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:49:46.546258 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485498 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:49:46.546258 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485501 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:49:46.546258 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485504 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:49:46.546258 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485506 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:49:46.546258 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485509 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:49:46.546258 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485512 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:49:46.546258 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485514 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:49:46.546258 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485517 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:49:46.546258 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485519 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:49:46.546258 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485521 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:49:46.546258 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485524 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:49:46.546258 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485526 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:49:46.546258 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485529 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:49:46.546258 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485531 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:49:46.546258 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485533 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:49:46.546258 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485536 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:49:46.546258 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485538 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:49:46.546258 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485541 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:49:46.546258 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485543 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:49:46.574652 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485545 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:49:46.574652 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485547 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:49:46.574652 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485550 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:49:46.574652 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485552 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:49:46.574652 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485555 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:49:46.574652 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485557 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:49:46.574652 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485560 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:49:46.574652 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485562 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:49:46.574652 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485565 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:49:46.574652 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485567 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:49:46.574652 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485571 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:49:46.574652 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485573 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:49:46.574652 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485577 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:49:46.574652 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.485579 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:49:46.574652 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485652 2564 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 18:49:46.574652 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485661 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 18:49:46.574652 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485670 2564 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 18:49:46.574652 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485677 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 18:49:46.574652 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485684 2564 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 18:49:46.574652 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485688 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 18:49:46.574652 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485694 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 18:49:46.547339 ip-10-0-135-81 systemd[1]: Started Kubernetes Kubelet. Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485699 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485702 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485705 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485717 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485720 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485723 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485726 2564 flags.go:64] FLAG: --cgroup-root="" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485729 2564 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485732 2564 flags.go:64] FLAG: --client-ca-file="" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485734 2564 flags.go:64] FLAG: --cloud-provider="external" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485737 2564 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485741 2564 flags.go:64] FLAG: --cluster-domain="" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485744 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485746 2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485749 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485753 2564 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485756 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485759 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485762 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485765 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485768 2564 flags.go:64] FLAG: --contention-profiling="false" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485772 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485775 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 18:49:46.576019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485779 2564 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485782 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485786 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485789 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485792 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485795 2564 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485797 2564 flags.go:64] FLAG: --enable-server="true" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485801 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485805 2564 flags.go:64] FLAG: --event-burst="100" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485808 2564 flags.go:64] FLAG: --event-qps="50" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485811 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485814 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485817 2564 flags.go:64] FLAG: --eviction-hard="" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485820 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485823 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485825 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485828 2564 flags.go:64] FLAG: --eviction-soft="" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485831 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485834 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485837 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485840 2564 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485843 2564 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485845 2564 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485848 2564 flags.go:64] FLAG: --feature-gates="" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485852 2564 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 18:49:46.584414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485854 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485857 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485860 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485863 2564 flags.go:64] FLAG: --healthz-port="10248" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485866 2564 flags.go:64] FLAG: --help="false" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485871 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-135-81.ec2.internal" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485874 2564 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485878 2564 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485881 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485885 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485888 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485890 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485894 2564 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485896 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485899 2564 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485903 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485906 2564 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485908 2564 flags.go:64] FLAG: --kube-reserved="" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485911 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485914 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485917 2564 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485919 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485922 2564 flags.go:64] FLAG: --lock-file="" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485924 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 18:49:46.586782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485927 2564 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485930 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485935 2564 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485937 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485940 2564 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485943 2564 flags.go:64] FLAG: --logging-format="text" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485945 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485948 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485951 2564 flags.go:64] FLAG: --manifest-url="" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485954 2564 flags.go:64] FLAG: --manifest-url-header="" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485958 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485961 2564 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485965 2564 flags.go:64] FLAG: --max-pods="110" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485970 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485972 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485976 2564 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485979 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485982 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485985 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485987 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485994 2564 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.485997 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486000 2564 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 18:49:46.726278 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486003 2564 flags.go:64] FLAG: --pod-cidr="" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486006 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486011 2564 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486014 2564 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486017 2564 flags.go:64] FLAG: --pods-per-core="0" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486019 2564 flags.go:64] FLAG: --port="10250" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486022 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486025 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08867e4ed96a42b7c" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486028 2564 flags.go:64] FLAG: --qos-reserved="" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486031 2564 flags.go:64] FLAG: --read-only-port="10255" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486033 2564 flags.go:64] FLAG: --register-node="true" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486036 2564 flags.go:64] FLAG: --register-with-taints="" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486040 2564 flags.go:64] FLAG: --registry-burst="10" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486042 2564 flags.go:64] FLAG: --registry-qps="5" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486045 2564 flags.go:64] FLAG: --reserved-cpus="" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486047 2564 flags.go:64] FLAG: --reserved-memory="" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486051 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486054 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486056 2564 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486059 2564 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486062 2564 flags.go:64] FLAG: --runonce="false" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486065 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486068 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486071 2564 flags.go:64] FLAG: --seccomp-default="false" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486076 2564 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 18:49:46.727686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486079 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486081 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486084 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486087 2564 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486090 2564 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486092 2564 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486095 2564 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486098 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486101 2564 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486104 2564 flags.go:64] FLAG: --system-cgroups="" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486106 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486112 2564 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486114 2564 flags.go:64] FLAG: --tls-cert-file="" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486117 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486121 2564 flags.go:64] FLAG: --tls-min-version="" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486124 2564 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486126 2564 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486129 2564 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486132 2564 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486135 2564 flags.go:64] FLAG: --v="2" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486138 2564 flags.go:64] FLAG: --version="false" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486142 2564 flags.go:64] FLAG: --vmodule="" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486145 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.486148 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486279 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:49:46.729063 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486284 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:49:46.731209 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486287 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:49:46.731209 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486290 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:49:46.731209 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486294 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:49:46.731209 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486297 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:49:46.731209 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486302 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:49:46.731209 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486306 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:49:46.731209 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486309 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:49:46.731209 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486311 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:49:46.731209 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486314 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:49:46.731209 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486316 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:49:46.731209 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486319 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:49:46.731209 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486321 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:49:46.731209 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486324 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:49:46.731209 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486327 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:49:46.731209 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486329 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:49:46.731209 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486332 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:49:46.731209 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486340 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:49:46.731209 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486344 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:49:46.731209 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486346 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:49:46.731209 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486349 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:49:46.732521 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486352 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:49:46.732521 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486354 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:49:46.732521 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486356 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:49:46.732521 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486359 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:49:46.732521 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486361 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:49:46.732521 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486363 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:49:46.732521 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486366 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:49:46.732521 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486368 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:49:46.732521 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486371 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:49:46.732521 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486373 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:49:46.732521 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486376 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:49:46.732521 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486378 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:49:46.732521 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486381 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:49:46.732521 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486383 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:49:46.732521 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486385 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:49:46.732521 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486388 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:49:46.732521 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486409 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:49:46.732521 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486417 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:49:46.732521 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486420 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:49:46.733862 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486423 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:49:46.733862 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486425 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:49:46.733862 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486428 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:49:46.733862 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486430 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:49:46.733862 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486433 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:49:46.733862 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486435 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:49:46.733862 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486439 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:49:46.733862 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486441 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:49:46.733862 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486443 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:49:46.733862 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486446 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:49:46.733862 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486454 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:49:46.733862 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486457 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:49:46.733862 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486459 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:49:46.733862 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486462 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:49:46.733862 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486464 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:49:46.733862 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486467 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:49:46.733862 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486469 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:49:46.733862 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486471 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:49:46.733862 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486474 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:49:46.733862 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486476 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:49:46.735186 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486478 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:49:46.735186 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486481 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:49:46.735186 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486483 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:49:46.735186 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486485 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:49:46.735186 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486488 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:49:46.735186 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486490 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:49:46.735186 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486493 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:49:46.735186 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486495 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:49:46.735186 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486498 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:49:46.735186 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486501 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:49:46.735186 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486505 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:49:46.735186 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486507 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:49:46.735186 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486510 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:49:46.735186 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486512 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:49:46.735186 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486514 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:49:46.735186 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486517 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:49:46.735186 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486519 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:49:46.735186 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486522 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:49:46.735186 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486524 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:49:46.735186 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486527 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:49:46.736538 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486529 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:49:46.736538 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486532 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:49:46.736538 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486534 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:49:46.736538 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486542 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:49:46.736538 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486545 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:49:46.736538 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486548 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:49:46.736538 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486550 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:49:46.736538 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486553 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:49:46.736538 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486556 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:49:46.736538 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486558 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:49:46.736538 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486560 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:49:46.736538 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486563 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:49:46.736538 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486565 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:49:46.736538 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486568 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:49:46.736538 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486570 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:49:46.736538 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486574 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:49:46.736538 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486577 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:49:46.736538 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.486580 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:49:46.738685 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.487049 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 18:49:46.738685 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.493771 2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 18:49:46.738685 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.493784 2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 18:49:46.738685 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.493801 2564 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 18:49:46.738685 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.493806 2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 18:49:46.738685 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493872 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:49:46.738685 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493877 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:49:46.738685 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493880 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:49:46.738685 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493883 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:49:46.738685 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493886 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:49:46.738685 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493888 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:49:46.738685 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493892 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:49:46.738685 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493894 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:49:46.738685 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493897 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:49:46.738685 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493899 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:49:46.739312 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493902 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:49:46.739312 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493904 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:49:46.739312 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493914 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:49:46.739312 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493917 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:49:46.739312 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493919 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:49:46.739312 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493922 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:49:46.739312 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493924 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:49:46.739312 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493927 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:49:46.739312 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493929 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:49:46.739312 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493932 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:49:46.739312 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493935 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:49:46.739312 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493938 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:49:46.739312 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493941 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:49:46.739312 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493943 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:49:46.739312 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493945 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:49:46.739312 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493948 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:49:46.739312 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493950 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:49:46.739312 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493953 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:49:46.739312 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493956 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:49:46.739312 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493958 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:49:46.740097 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493961 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:49:46.740097 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493963 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:49:46.740097 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493970 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:49:46.740097 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493973 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:49:46.740097 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493975 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:49:46.740097 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493978 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:49:46.740097 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493980 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:49:46.740097 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493982 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:49:46.740097 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493985 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:49:46.740097 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493987 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:49:46.740097 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493989 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:49:46.740097 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493993 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:49:46.740097 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.493997 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:49:46.740097 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494000 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:49:46.740097 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494003 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:49:46.740097 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494011 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:49:46.740097 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494014 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:49:46.740097 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494016 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:49:46.740097 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494019 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:49:46.740838 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494021 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:49:46.740838 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494024 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:49:46.740838 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494026 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:49:46.740838 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494029 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:49:46.740838 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494031 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:49:46.740838 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494035 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:49:46.740838 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494044 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:49:46.740838 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494047 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:49:46.740838 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494049 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:49:46.740838 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494052 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:49:46.740838 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494054 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:49:46.740838 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494057 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:49:46.740838 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494060 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:49:46.740838 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494062 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:49:46.740838 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494065 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:49:46.740838 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494068 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:49:46.740838 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494070 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:49:46.740838 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494073 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:49:46.740838 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494075 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:49:46.740838 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494078 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:49:46.741617 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494080 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:49:46.741617 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494082 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:49:46.741617 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494085 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:49:46.741617 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494087 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:49:46.741617 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494089 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:49:46.741617 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494092 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:49:46.741617 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494094 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:49:46.741617 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494098 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:49:46.741617 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494101 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:49:46.741617 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494109 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:49:46.741617 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494112 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:49:46.741617 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494115 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:49:46.741617 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494117 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:49:46.741617 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494120 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:49:46.741617 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494122 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:49:46.741617 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494125 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:49:46.741617 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494128 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:49:46.741617 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494131 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:49:46.741617 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494134 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:49:46.742351 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494136 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:49:46.742351 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494138 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:49:46.742351 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494141 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:49:46.742351 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494144 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:49:46.742351 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494147 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:49:46.742351 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494149 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:49:46.742351 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494152 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:49:46.742351 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494154 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:49:46.742351 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494157 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:49:46.742351 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494159 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:49:46.742351 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494162 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:49:46.742351 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.494167 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 18:49:46.742351 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494328 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 18:49:46.742351 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494333 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 18:49:46.742351 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494336 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 18:49:46.742939 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494339 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 18:49:46.742939 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494342 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 18:49:46.742939 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494345 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 18:49:46.742939 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494347 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 18:49:46.742939 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494350 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 18:49:46.742939 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494352 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 18:49:46.742939 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494355 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 18:49:46.742939 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494357 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 18:49:46.742939 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494366 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 18:49:46.742939 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494369 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 18:49:46.742939 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494372 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 18:49:46.742939 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494374 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 18:49:46.742939 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494377 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 18:49:46.742939 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494380 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 18:49:46.742939 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494382 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 18:49:46.742939 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494384 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 18:49:46.742939 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494387 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 18:49:46.742939 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494389 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 18:49:46.742939 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494410 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 18:49:46.742939 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494414 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 18:49:46.743722 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494416 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 18:49:46.743722 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494421 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 18:49:46.743722 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494423 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 18:49:46.743722 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494426 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 18:49:46.743722 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494428 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 18:49:46.743722 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494431 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 18:49:46.743722 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494433 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 18:49:46.743722 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494436 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 18:49:46.743722 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494438 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 18:49:46.743722 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494440 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 18:49:46.743722 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494443 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 18:49:46.743722 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494445 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 18:49:46.743722 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494448 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 18:49:46.743722 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494450 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 18:49:46.743722 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494453 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 18:49:46.743722 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494455 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 18:49:46.743722 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494457 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 18:49:46.743722 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494460 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 18:49:46.743722 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494463 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 18:49:46.743722 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494465 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 18:49:46.744524 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494468 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 18:49:46.744524 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494476 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 18:49:46.744524 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494479 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 18:49:46.744524 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494483 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 18:49:46.744524 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494486 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 18:49:46.744524 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494488 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 18:49:46.744524 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494490 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 18:49:46.744524 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494493 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 18:49:46.744524 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494495 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 18:49:46.744524 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494497 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 18:49:46.744524 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494500 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 18:49:46.744524 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494502 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 18:49:46.744524 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494504 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 18:49:46.744524 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494508 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 18:49:46.744524 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494511 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 18:49:46.744524 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494513 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 18:49:46.744524 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494515 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 18:49:46.744524 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494518 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 18:49:46.744524 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494520 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 18:49:46.744524 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494523 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 18:49:46.745342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494525 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 18:49:46.745342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494528 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 18:49:46.745342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494530 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 18:49:46.745342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494532 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 18:49:46.745342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494535 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 18:49:46.745342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494537 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 18:49:46.745342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494539 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 18:49:46.745342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494542 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 18:49:46.745342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494545 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 18:49:46.745342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494547 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 18:49:46.745342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494549 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 18:49:46.745342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494553 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 18:49:46.745342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494557 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 18:49:46.745342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494559 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 18:49:46.745342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494568 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 18:49:46.745342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494571 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 18:49:46.745342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494574 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 18:49:46.745342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494576 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 18:49:46.745342 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494580 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 18:49:46.746100 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494583 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 18:49:46.746100 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494586 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 07 18:49:46.746100 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494589 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 18:49:46.746100 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494591 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 18:49:46.746100 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494595 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 18:49:46.746100 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494598 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 18:49:46.746100 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494602 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 18:49:46.746100 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494605 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 18:49:46.746100 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494608 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 18:49:46.746100 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494610 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 18:49:46.746100 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494613 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 18:49:46.746100 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494616 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 18:49:46.746100 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494619 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 18:49:46.746100 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494621 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 18:49:46.746100 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494623 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 18:49:46.746100 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494626 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 18:49:46.746100 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:46.494628 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 18:49:46.746748 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.494633 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 18:49:46.746748 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.494732 2564 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 18:49:46.746748 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.496684 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 18:49:46.746748 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.497918 2564 server.go:1019] "Starting client certificate rotation" Jul 07 18:49:46.746748 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.498025 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 18:49:46.746748 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.498706 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 18:49:46.746748 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.500034 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 18:49:46.746748 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.505007 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 18:49:46.746748 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.513944 2564 log.go:25] "Validated CRI v1 runtime API" Jul 07 18:49:46.746748 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.518142 2564 log.go:25] "Validated CRI v1 image API" Jul 07 18:49:46.746748 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.519547 2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 18:49:46.746748 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.524271 2564 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 aa8763d7-7e27-4763-be59-0d9373edc288:/dev/nvme0n1p4 ff3b6201-176b-4d2b-9227-d9e9c4e6f36b:/dev/nvme0n1p3] Jul 07 18:49:46.747218 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.524284 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 18:49:46.747268 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.528902 2564 manager.go:216] Machine: {Timestamp:2026-07-07 18:49:46.527724717 +0000 UTC m=+0.264162308 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3195146 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24cb4bf8a526db32e9c9e9ae26affe SystemUUID:ec24cb4b-f8a5-26db-32e9-c9e9ae26affe BootID:d498fa3c-2fa8-4bef-827d-92fbe10a2b06 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a8:6e:fb:ab:65 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a8:6e:fb:ab:65 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:d9:89:57:07:03 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 18:49:46.747268 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.529005 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 18:49:46.747268 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.529074 2564 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 18:49:46.747268 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.530846 2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 18:49:46.747268 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.530873 2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-81.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 18:49:46.747268 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.531494 2564 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 18:49:46.747268 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.531503 2564 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 18:49:46.747268 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.531510 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.532046 2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.533246 2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.534734 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.537064 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.537174 2564 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.537201 2564 kubelet.go:486] "Attempting to sync node with API server" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.537214 2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.537234 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.537245 2564 kubelet.go:398] "Adding apiserver pod source" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.537257 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.537618 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.537636 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.539479 2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.541247 2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.541273 2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.542237 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.542251 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.542256 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.542262 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.542270 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.542278 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.542292 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.542297 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.542304 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.542310 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.542319 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.542324 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.543112 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.543119 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.546655 2564 server.go:1268] "Started kubelet" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.546981 2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.547047 2564 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.547179 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.548827 2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.550923 2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-81.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:46.551946 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-81.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.551961 2564 server.go:310] "Adding debug handlers to kubelet server" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:46.551977 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 18:49:46.747635 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:46.552546 2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:46.553866 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-81.ec2.internal.18c0167edbb74470 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-81.ec2.internal,UID:ip-10-0-135-81.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-81.ec2.internal,},FirstTimestamp:2026-07-07 18:49:46.546627696 +0000 UTC m=+0.283065287,LastTimestamp:2026-07-07 18:49:46.546627696 +0000 UTC m=+0.283065287,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-81.ec2.internal,}" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.557078 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.557086 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:46.557149 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-81.ec2.internal\" not found" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.557644 2564 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.557657 2564 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.557690 2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.557797 2564 reconstruct.go:97] "Volume reconstruction finished" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.557807 2564 reconciler.go:29] "Reconciler: start to sync state" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.558062 2564 factory.go:55] Registering systemd factory Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.558077 2564 factory.go:223] Registration of the systemd container factory successfully Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.558269 2564 factory.go:153] Registering CRI-O factory Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.558283 2564 factory.go:223] Registration of the crio container factory successfully Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.558329 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.558350 2564 factory.go:103] Registering Raw factory Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.558363 2564 manager.go:1196] Started watching for new ooms in manager Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.558732 2564 manager.go:349] Starting recovery of all containers Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.561431 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-48dwx" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:46.568273 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:46.568304 2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-81.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.578294 2564 manager.go:354] Recovery completed Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.595338 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-48dwx" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:46.597045 2564 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.601058 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.602811 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.602829 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.602838 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.603303 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.603311 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.603324 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.605292 2564 policy_none.go:49] "None policy: Start" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.605303 2564 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.605313 2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 18:49:46.749159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.607242 2564 policy_none.go:47] "Start" Jul 07 18:49:46.750524 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.607689 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:49:46.750524 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:46.607805 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-81.ec2.internal.18c0167edf10b7c4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-81.ec2.internal,UID:ip-10-0-135-81.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-81.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-81.ec2.internal,},FirstTimestamp:2026-07-07 18:49:46.602821572 +0000 UTC m=+0.339259163,LastTimestamp:2026-07-07 18:49:46.602821572 +0000 UTC m=+0.339259163,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-81.ec2.internal,}" Jul 07 18:49:46.750524 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.608347 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:49:46.750524 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.608373 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:49:46.750524 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.608385 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:49:46.750524 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.639892 2564 manager.go:337] "Starting Device Plugin manager" Jul 07 18:49:46.750524 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:46.640015 2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 18:49:46.750524 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.640028 2564 server.go:87] "Starting device plugin registration server" Jul 07 18:49:46.750524 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.640185 2564 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 18:49:46.750524 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.640193 2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 18:49:46.750524 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.640257 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 18:49:46.750524 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.640327 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 18:49:46.750524 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.640335 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 18:49:46.750524 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:46.640743 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 18:49:46.750524 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:46.640812 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-81.ec2.internal\" not found" Jul 07 18:49:46.750524 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.740566 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:49:46.750524 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.741353 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:49:46.750524 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.741380 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:49:46.750524 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.741389 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:49:46.750524 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.741428 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-81.ec2.internal" Jul 07 18:49:46.751288 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.750934 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-81.ec2.internal" Jul 07 18:49:46.751288 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:46.750954 2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-81.ec2.internal\": node \"ip-10-0-135-81.ec2.internal\" not found" Jul 07 18:49:46.763668 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:46.763654 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-81.ec2.internal\" not found" Jul 07 18:49:46.798930 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.798907 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 18:49:46.799992 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.799974 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 18:49:46.800050 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.800004 2564 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 18:49:46.800050 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.800021 2564 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 18:49:46.800050 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:46.800046 2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 18:49:46.802540 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:46.802523 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 18:49:48.630920 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:48.630905 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-81.ec2.internal\" not found" Jul 07 18:49:48.667063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.667046 2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-81.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-81.ec2.internal"] Jul 07 18:49:48.667120 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.667111 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:49:48.667921 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.667906 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:49:48.667982 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.667932 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:49:48.667982 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.667944 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:49:48.670237 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.670224 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:49:48.670304 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.670290 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-81.ec2.internal" Jul 07 18:49:48.670346 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.670332 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:49:48.670843 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.670825 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:49:48.670919 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.670855 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:49:48.670919 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.670865 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:49:48.670919 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.670825 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:49:48.671019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.670926 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:49:48.671019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.670939 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:49:48.673131 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.673118 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-81.ec2.internal" Jul 07 18:49:48.673178 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.673141 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 18:49:48.673764 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.673749 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeHasSufficientMemory" Jul 07 18:49:48.673821 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.673776 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 18:49:48.673821 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.673790 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeHasSufficientPID" Jul 07 18:49:48.694659 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:48.694644 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-81.ec2.internal\" not found" node="ip-10-0-135-81.ec2.internal" Jul 07 18:49:48.698718 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:48.698703 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-81.ec2.internal\" not found" node="ip-10-0-135-81.ec2.internal" Jul 07 18:49:48.726868 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.726825 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ee8c04a12a35cab5864ba50e62332606-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-81.ec2.internal\" (UID: \"ee8c04a12a35cab5864ba50e62332606\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-81.ec2.internal" Jul 07 18:49:48.726868 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.726847 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee8c04a12a35cab5864ba50e62332606-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-81.ec2.internal\" (UID: \"ee8c04a12a35cab5864ba50e62332606\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-81.ec2.internal" Jul 07 18:49:48.726868 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.726862 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c7bff29c072ef9b7b9d4f7fce9f2d5cc-config\") pod \"kube-apiserver-proxy-ip-10-0-135-81.ec2.internal\" (UID: \"c7bff29c072ef9b7b9d4f7fce9f2d5cc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-81.ec2.internal" Jul 07 18:49:48.731409 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:48.731391 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-81.ec2.internal\" not found" Jul 07 18:49:48.827694 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.827681 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ee8c04a12a35cab5864ba50e62332606-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-81.ec2.internal\" (UID: \"ee8c04a12a35cab5864ba50e62332606\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-81.ec2.internal" Jul 07 18:49:48.827766 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.827724 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee8c04a12a35cab5864ba50e62332606-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-81.ec2.internal\" (UID: \"ee8c04a12a35cab5864ba50e62332606\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-81.ec2.internal" Jul 07 18:49:48.827766 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.827748 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ee8c04a12a35cab5864ba50e62332606-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-81.ec2.internal\" (UID: \"ee8c04a12a35cab5864ba50e62332606\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-81.ec2.internal" Jul 07 18:49:48.827851 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.827772 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c7bff29c072ef9b7b9d4f7fce9f2d5cc-config\") pod \"kube-apiserver-proxy-ip-10-0-135-81.ec2.internal\" (UID: \"c7bff29c072ef9b7b9d4f7fce9f2d5cc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-81.ec2.internal" Jul 07 18:49:48.827851 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.827778 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ee8c04a12a35cab5864ba50e62332606-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-81.ec2.internal\" (UID: \"ee8c04a12a35cab5864ba50e62332606\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-81.ec2.internal" Jul 07 18:49:48.827851 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.827793 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c7bff29c072ef9b7b9d4f7fce9f2d5cc-config\") pod \"kube-apiserver-proxy-ip-10-0-135-81.ec2.internal\" (UID: \"c7bff29c072ef9b7b9d4f7fce9f2d5cc\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-81.ec2.internal" Jul 07 18:49:48.832415 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:48.832402 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-81.ec2.internal\" not found" Jul 07 18:49:48.932883 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:48.932847 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-81.ec2.internal\" not found" Jul 07 18:49:48.998046 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:48.998000 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-81.ec2.internal" Jul 07 18:49:49.001585 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:49.001569 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-81.ec2.internal" Jul 07 18:49:49.033568 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:49.033545 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-81.ec2.internal\" not found" Jul 07 18:49:49.134020 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:49.133990 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-81.ec2.internal\" not found" Jul 07 18:49:49.234577 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:49.234551 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-81.ec2.internal\" not found" Jul 07 18:49:49.264757 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:49.264697 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 18:49:49.264846 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:49.264810 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 18:49:49.335078 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:49.335060 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-81.ec2.internal\" not found" Jul 07 18:49:49.365438 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:49.365410 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 18:44:48 +0000 UTC" deadline="2027-12-31 05:07:54.452636 +0000 UTC" Jul 07 18:49:49.365438 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:49.365434 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12994h18m5.087204952s" Jul 07 18:49:49.435946 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:49.435927 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-81.ec2.internal\" not found" Jul 07 18:49:49.457365 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:49.457346 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 18:49:49.518576 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:49.518527 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 18:49:49.536997 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:49.536977 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-81.ec2.internal\" not found" Jul 07 18:49:49.592974 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:49.592951 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7bff29c072ef9b7b9d4f7fce9f2d5cc.slice/crio-60e1d56b1a9581000f6e77cf900fd5fedaeb9abc018f814ef5426c700184fd32 WatchSource:0}: Error finding container 60e1d56b1a9581000f6e77cf900fd5fedaeb9abc018f814ef5426c700184fd32: Status 404 returned error can't find the container with id 60e1d56b1a9581000f6e77cf900fd5fedaeb9abc018f814ef5426c700184fd32 Jul 07 18:49:49.600881 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:49.600859 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 18:49:49.606685 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:49.606660 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee8c04a12a35cab5864ba50e62332606.slice/crio-e74fb38807691701da6e6a5649ad2c24cb6a51ca16456924f228bece8363d5bf WatchSource:0}: Error finding container e74fb38807691701da6e6a5649ad2c24cb6a51ca16456924f228bece8363d5bf: Status 404 returned error can't find the container with id e74fb38807691701da6e6a5649ad2c24cb6a51ca16456924f228bece8363d5bf Jul 07 18:49:49.637766 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:49.637749 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-81.ec2.internal\" not found" Jul 07 18:49:49.738190 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:49.738175 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-81.ec2.internal\" not found" Jul 07 18:49:49.838926 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:49.838873 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-81.ec2.internal\" not found" Jul 07 18:49:49.869272 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:49.869252 2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 18:49:49.924093 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:49.924064 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-81.ec2.internal" Jul 07 18:49:49.936281 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:49.936260 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 18:49:49.937107 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:49.937092 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-81.ec2.internal" Jul 07 18:49:49.948268 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:49.948247 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 18:49:50.305187 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.305157 2564 apiserver.go:52] "Watching apiserver" Jul 07 18:49:50.315479 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.315318 2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 18:49:50.315708 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.315683 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh","openshift-dns/node-resolver-8bhnt","openshift-image-registry/node-ca-qkhdg","openshift-multus/multus-additional-cni-plugins-6cb6g","openshift-multus/network-metrics-daemon-ccjd5","openshift-network-operator/iptables-alerter-wlrjs","kube-system/konnectivity-agent-w4rcb","kube-system/kube-apiserver-proxy-ip-10-0-135-81.ec2.internal","openshift-cluster-node-tuning-operator/tuned-qtjjp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-81.ec2.internal","openshift-multus/multus-kk8zs","openshift-network-diagnostics/network-check-target-mkg6r","openshift-ovn-kubernetes/ovnkube-node-vkzgb","kube-system/global-pull-secret-syncer-ffm5c"] Jul 07 18:49:50.318304 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.318275 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w4rcb" Jul 07 18:49:50.320740 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.320717 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 18:49:50.320848 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.320781 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.320848 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.320814 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-f5xf6\"" Jul 07 18:49:50.321050 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.321024 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 18:49:50.323027 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.322761 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6bh9s\"" Jul 07 18:49:50.323143 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.323059 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 18:49:50.323143 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.323116 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qkhdg" Jul 07 18:49:50.323513 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.323353 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 18:49:50.325076 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.324182 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 18:49:50.325076 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.324258 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 18:49:50.325076 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.324455 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 18:49:50.325076 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.324468 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 18:49:50.325076 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.324603 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 18:49:50.326260 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.326231 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.328476 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.328459 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:49:50.328568 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:50.328538 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjd5" podUID="536dbfa9-2261-4932-a922-e11a2a764168" Jul 07 18:49:50.332946 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.332924 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wlrjs" Jul 07 18:49:50.334808 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.334786 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/330e5b54-ff72-4853-b885-9a3327275f99-konnectivity-ca\") pod \"konnectivity-agent-w4rcb\" (UID: \"330e5b54-ff72-4853-b885-9a3327275f99\") " pod="kube-system/konnectivity-agent-w4rcb" Jul 07 18:49:50.334904 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.334822 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-host-slash\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.334904 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.334844 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8821d59a-1e1c-43d8-80d8-3513deb7478b-host\") pod \"node-ca-qkhdg\" (UID: \"8821d59a-1e1c-43d8-80d8-3513deb7478b\") " pod="openshift-image-registry/node-ca-qkhdg" Jul 07 18:49:50.334904 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.334865 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-run-systemd\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.334904 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.334887 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.335097 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.334934 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-ovn-node-metrics-cert\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.335097 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.334986 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-ovnkube-script-lib\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.335097 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335014 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52g85\" (UniqueName: \"kubernetes.io/projected/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-kube-api-access-52g85\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.335097 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335038 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-os-release\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.335097 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335061 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-cni-binary-copy\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.335097 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335084 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.335097 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335087 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-h4jpk\"" Jul 07 18:49:50.335881 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335123 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-host-run-netns\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.335881 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335150 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-node-log\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.335881 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335170 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-host-cni-bin\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.335881 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335191 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-host-cni-netd\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.335881 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335229 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8821d59a-1e1c-43d8-80d8-3513deb7478b-serviceca\") pod \"node-ca-qkhdg\" (UID: \"8821d59a-1e1c-43d8-80d8-3513deb7478b\") " pod="openshift-image-registry/node-ca-qkhdg" Jul 07 18:49:50.335881 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335264 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.335881 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335287 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-ovnkube-config\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.335881 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335301 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd6wh\" (UniqueName: \"kubernetes.io/projected/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-kube-api-access-hd6wh\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.335881 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335322 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-run-ovn\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.335881 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335349 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-log-socket\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.335881 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335373 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-system-cni-dir\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.335881 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335395 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.335881 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335421 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-run-openvswitch\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.336570 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335455 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-host-kubelet\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.336570 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335473 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-systemd-units\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.336570 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335507 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xwn7\" (UniqueName: \"kubernetes.io/projected/8821d59a-1e1c-43d8-80d8-3513deb7478b-kube-api-access-7xwn7\") pod \"node-ca-qkhdg\" (UID: \"8821d59a-1e1c-43d8-80d8-3513deb7478b\") " pod="openshift-image-registry/node-ca-qkhdg" Jul 07 18:49:50.336570 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335527 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.336570 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335533 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/330e5b54-ff72-4853-b885-9a3327275f99-agent-certs\") pod \"konnectivity-agent-w4rcb\" (UID: \"330e5b54-ff72-4853-b885-9a3327275f99\") " pod="kube-system/konnectivity-agent-w4rcb" Jul 07 18:49:50.336570 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335556 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-var-lib-openvswitch\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.336570 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335579 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-etc-openvswitch\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.336570 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335604 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-host-run-ovn-kubernetes\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.336570 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335641 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-env-overrides\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.336570 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.335667 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-cnibin\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.337536 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.337518 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 18:49:50.337929 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.337913 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.340334 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.340267 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8bhnt" Jul 07 18:49:50.342574 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.342554 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:49:50.342662 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:50.342620 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffm5c" podUID="570bed30-5e16-475c-9cc4-4e5af148c2e5" Jul 07 18:49:50.344878 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.344861 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.347333 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.347249 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:49:50.347333 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:50.347305 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mkg6r" podUID="88340235-0f7e-4e90-bf52-86b3f04aa9ff" Jul 07 18:49:50.376958 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.376937 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 18:49:50.377049 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.376989 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 18:49:50.377102 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.376939 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nrgnd\"" Jul 07 18:49:50.381969 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.381361 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cp4mh\"" Jul 07 18:49:50.381969 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.381381 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 18:49:50.381969 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.381558 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 18:49:50.381969 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.381580 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 18:49:50.382398 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.382385 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 18:49:50.383348 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.383330 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 18:49:50.390108 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.390090 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 18:49:50.390722 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.390286 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:49:50.390722 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.390487 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:49:50.390722 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.390569 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 18:49:50.390722 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.390573 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 18:49:50.390722 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.390691 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 18:49:50.390722 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.390696 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 18:49:50.391028 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.390760 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 18:49:50.391028 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.390976 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 18:49:50.391116 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.391038 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 18:49:50.391116 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.391098 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nr8dh\"" Jul 07 18:49:50.391228 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.391140 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pszlz\"" Jul 07 18:49:50.391282 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.391263 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sfrvp\"" Jul 07 18:49:50.391727 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.391350 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lmzcr\"" Jul 07 18:49:50.401304 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.401286 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 18:49:50.406633 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.406616 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 18:49:50.425963 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.425903 2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 18:49:50.435881 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.435860 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-os-release\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.435991 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.435891 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-cni-binary-copy\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.435991 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.435919 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-var-lib-kubelet\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.435991 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.435941 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2f1fd794-cf7b-4072-8df4-1f951b7f2f4c-socket-dir\") pod \"aws-ebs-csi-driver-node-zmzfh\" (UID: \"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.435991 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.435963 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-cnibin\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.435991 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.435985 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-node-log\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.436249 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436007 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-host-cni-bin\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.436249 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436006 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-os-release\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.436249 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436029 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-host-cni-netd\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.436249 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436055 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.436249 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436080 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dcc43c9-3758-47a3-aa8d-278166547dad-tmp-dir\") pod \"node-resolver-8bhnt\" (UID: \"7dcc43c9-3758-47a3-aa8d-278166547dad\") " pod="openshift-dns/node-resolver-8bhnt" Jul 07 18:49:50.436249 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436101 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-os-release\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.436249 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436122 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-hostroot\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.436249 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436156 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-multus-daemon-config\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.436249 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436182 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd6wh\" (UniqueName: \"kubernetes.io/projected/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-kube-api-access-hd6wh\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.436249 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436221 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs\") pod \"network-metrics-daemon-ccjd5\" (UID: \"536dbfa9-2261-4932-a922-e11a2a764168\") " pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:49:50.436249 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436245 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5lt5\" (UniqueName: \"kubernetes.io/projected/4be6648f-5fd4-4351-b06b-cee32b5f028f-kube-api-access-s5lt5\") pod \"iptables-alerter-wlrjs\" (UID: \"4be6648f-5fd4-4351-b06b-cee32b5f028f\") " pod="openshift-network-operator/iptables-alerter-wlrjs" Jul 07 18:49:50.436738 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436266 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/570bed30-5e16-475c-9cc4-4e5af148c2e5-dbus\") pod \"global-pull-secret-syncer-ffm5c\" (UID: \"570bed30-5e16-475c-9cc4-4e5af148c2e5\") " pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:49:50.436738 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436288 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-host-run-multus-certs\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.436738 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436313 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-run-ovn\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.436738 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436335 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-log-socket\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.436738 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436358 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lfqs\" (UniqueName: \"kubernetes.io/projected/536dbfa9-2261-4932-a922-e11a2a764168-kube-api-access-2lfqs\") pod \"network-metrics-daemon-ccjd5\" (UID: \"536dbfa9-2261-4932-a922-e11a2a764168\") " pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:49:50.436738 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436378 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-run\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.436738 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436400 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret\") pod \"global-pull-secret-syncer-ffm5c\" (UID: \"570bed30-5e16-475c-9cc4-4e5af148c2e5\") " pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:49:50.436738 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436424 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-system-cni-dir\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.436738 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436454 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-run-openvswitch\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.436738 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436476 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-etc-sysctl-d\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.436738 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436496 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-host\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.436738 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436525 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bh4z\" (UniqueName: \"kubernetes.io/projected/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-kube-api-access-5bh4z\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.436738 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436549 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xwn7\" (UniqueName: \"kubernetes.io/projected/8821d59a-1e1c-43d8-80d8-3513deb7478b-kube-api-access-7xwn7\") pod \"node-ca-qkhdg\" (UID: \"8821d59a-1e1c-43d8-80d8-3513deb7478b\") " pod="openshift-image-registry/node-ca-qkhdg" Jul 07 18:49:50.436738 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436555 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-cni-binary-copy\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.437372 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436571 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-etc-modprobe-d\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.437372 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436596 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-etc-sysctl-conf\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.437372 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436633 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2f1fd794-cf7b-4072-8df4-1f951b7f2f4c-device-dir\") pod \"aws-ebs-csi-driver-node-zmzfh\" (UID: \"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.437372 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436782 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-host-run-netns\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.437372 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436804 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-run-ovn\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.437372 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436810 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/330e5b54-ff72-4853-b885-9a3327275f99-agent-certs\") pod \"konnectivity-agent-w4rcb\" (UID: \"330e5b54-ff72-4853-b885-9a3327275f99\") " pod="kube-system/konnectivity-agent-w4rcb" Jul 07 18:49:50.437372 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436849 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-etc-openvswitch\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.437372 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436850 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-run-openvswitch\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.437372 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436884 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-host-cni-bin\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.437372 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436901 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-etc-openvswitch\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.437372 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436925 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-host-cni-netd\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.437372 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436969 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-log-socket\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.437372 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.436999 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-host-run-ovn-kubernetes\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.437372 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437025 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-env-overrides\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.437963 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437038 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-node-log\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.437963 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437087 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-host-run-ovn-kubernetes\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.437963 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437144 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-cnibin\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.437963 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437206 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7dcc43c9-3758-47a3-aa8d-278166547dad-hosts-file\") pod \"node-resolver-8bhnt\" (UID: \"7dcc43c9-3758-47a3-aa8d-278166547dad\") " pod="openshift-dns/node-resolver-8bhnt" Jul 07 18:49:50.437963 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437245 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-cnibin\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.437963 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437307 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.437963 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437311 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-multus-socket-dir-parent\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.437963 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437352 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52v7l\" (UniqueName: \"kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l\") pod \"network-check-target-mkg6r\" (UID: \"88340235-0f7e-4e90-bf52-86b3f04aa9ff\") " pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:49:50.437963 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437347 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 18:49:50.437963 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437394 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-host-slash\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.437963 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437420 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-sys\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.437963 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437456 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-host-slash\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.437963 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437467 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-env-overrides\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.437963 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437507 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-host-run-k8s-cni-cncf-io\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.438623 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437555 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-host-var-lib-cni-bin\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.438623 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437582 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-run-systemd\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.438623 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437607 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.438623 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437629 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52g85\" (UniqueName: \"kubernetes.io/projected/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-kube-api-access-52g85\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.438623 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437636 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-run-systemd\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.438623 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437654 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.438623 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437676 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2f1fd794-cf7b-4072-8df4-1f951b7f2f4c-registration-dir\") pod \"aws-ebs-csi-driver-node-zmzfh\" (UID: \"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.438623 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437694 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.438623 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.437698 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-host-var-lib-kubelet\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.438623 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438031 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-multus-conf-dir\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.438623 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438063 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-host-run-netns\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.438623 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438092 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8821d59a-1e1c-43d8-80d8-3513deb7478b-serviceca\") pod \"node-ca-qkhdg\" (UID: \"8821d59a-1e1c-43d8-80d8-3513deb7478b\") " pod="openshift-image-registry/node-ca-qkhdg" Jul 07 18:49:50.438623 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438116 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4be6648f-5fd4-4351-b06b-cee32b5f028f-host-slash\") pod \"iptables-alerter-wlrjs\" (UID: \"4be6648f-5fd4-4351-b06b-cee32b5f028f\") " pod="openshift-network-operator/iptables-alerter-wlrjs" Jul 07 18:49:50.438623 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438141 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2f1fd794-cf7b-4072-8df4-1f951b7f2f4c-etc-selinux\") pod \"aws-ebs-csi-driver-node-zmzfh\" (UID: \"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.439342 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438159 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-host-run-netns\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.439342 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438165 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-ovnkube-config\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.439342 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438115 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.439342 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438188 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-cni-binary-copy\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.439342 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438609 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8821d59a-1e1c-43d8-80d8-3513deb7478b-serviceca\") pod \"node-ca-qkhdg\" (UID: \"8821d59a-1e1c-43d8-80d8-3513deb7478b\") " pod="openshift-image-registry/node-ca-qkhdg" Jul 07 18:49:50.439342 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438635 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-ovnkube-config\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.439342 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438648 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-system-cni-dir\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.439342 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438688 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.439342 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438715 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d6gx\" (UniqueName: \"kubernetes.io/projected/2f1fd794-cf7b-4072-8df4-1f951b7f2f4c-kube-api-access-5d6gx\") pod \"aws-ebs-csi-driver-node-zmzfh\" (UID: \"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.439342 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438737 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-multus-cni-dir\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.439342 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438793 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-host-kubelet\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.439342 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438818 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-systemd-units\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.439342 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438841 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d25dd7ba-d534-4f05-b29c-8162b009dfff-etc-tuned\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.439342 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438863 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d25dd7ba-d534-4f05-b29c-8162b009dfff-tmp\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.439988 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438886 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2f1fd794-cf7b-4072-8df4-1f951b7f2f4c-sys-fs\") pod \"aws-ebs-csi-driver-node-zmzfh\" (UID: \"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.439988 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438910 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4be6648f-5fd4-4351-b06b-cee32b5f028f-iptables-alerter-script\") pod \"iptables-alerter-wlrjs\" (UID: \"4be6648f-5fd4-4351-b06b-cee32b5f028f\") " pod="openshift-network-operator/iptables-alerter-wlrjs" Jul 07 18:49:50.439988 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438974 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-etc-systemd\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.439988 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.438997 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-lib-modules\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.439988 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.439011 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.439988 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.439018 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkk7r\" (UniqueName: \"kubernetes.io/projected/7dcc43c9-3758-47a3-aa8d-278166547dad-kube-api-access-lkk7r\") pod \"node-resolver-8bhnt\" (UID: \"7dcc43c9-3758-47a3-aa8d-278166547dad\") " pod="openshift-dns/node-resolver-8bhnt" Jul 07 18:49:50.439988 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.439057 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-host-kubelet\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.439988 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.439057 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-var-lib-openvswitch\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.439988 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.439091 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-host-var-lib-cni-multus\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.439988 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.439096 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-var-lib-openvswitch\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.439988 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.439116 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/330e5b54-ff72-4853-b885-9a3327275f99-konnectivity-ca\") pod \"konnectivity-agent-w4rcb\" (UID: \"330e5b54-ff72-4853-b885-9a3327275f99\") " pod="kube-system/konnectivity-agent-w4rcb" Jul 07 18:49:50.439988 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.439167 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-systemd-units\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.439988 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.439223 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-system-cni-dir\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.439988 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.439252 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8821d59a-1e1c-43d8-80d8-3513deb7478b-host\") pod \"node-ca-qkhdg\" (UID: \"8821d59a-1e1c-43d8-80d8-3513deb7478b\") " pod="openshift-image-registry/node-ca-qkhdg" Jul 07 18:49:50.440586 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.439277 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-etc-sysconfig\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.440586 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.439300 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-etc-kubernetes\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.440586 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.439343 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6tlr\" (UniqueName: \"kubernetes.io/projected/d25dd7ba-d534-4f05-b29c-8162b009dfff-kube-api-access-g6tlr\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.440586 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.439368 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2f1fd794-cf7b-4072-8df4-1f951b7f2f4c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zmzfh\" (UID: \"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.440586 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.439390 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/570bed30-5e16-475c-9cc4-4e5af148c2e5-kubelet-config\") pod \"global-pull-secret-syncer-ffm5c\" (UID: \"570bed30-5e16-475c-9cc4-4e5af148c2e5\") " pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:49:50.440586 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.439415 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-etc-kubernetes\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.440586 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.439441 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-ovn-node-metrics-cert\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.440586 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.439465 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-ovnkube-script-lib\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.440586 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.439605 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/330e5b54-ff72-4853-b885-9a3327275f99-konnectivity-ca\") pod \"konnectivity-agent-w4rcb\" (UID: \"330e5b54-ff72-4853-b885-9a3327275f99\") " pod="kube-system/konnectivity-agent-w4rcb" Jul 07 18:49:50.440586 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.439663 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8821d59a-1e1c-43d8-80d8-3513deb7478b-host\") pod \"node-ca-qkhdg\" (UID: \"8821d59a-1e1c-43d8-80d8-3513deb7478b\") " pod="openshift-image-registry/node-ca-qkhdg" Jul 07 18:49:50.440586 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.439971 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-ovnkube-script-lib\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.440586 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.440244 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-6ddh7" Jul 07 18:49:50.443327 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.443270 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-ovn-node-metrics-cert\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.443493 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.443473 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/330e5b54-ff72-4853-b885-9a3327275f99-agent-certs\") pod \"konnectivity-agent-w4rcb\" (UID: \"330e5b54-ff72-4853-b885-9a3327275f99\") " pod="kube-system/konnectivity-agent-w4rcb" Jul 07 18:49:50.454317 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.454300 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-6ddh7" Jul 07 18:49:50.456606 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.456556 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd6wh\" (UniqueName: \"kubernetes.io/projected/b8a9c668-2d9f-4c07-9b0e-e9e57c20a796-kube-api-access-hd6wh\") pod \"multus-additional-cni-plugins-6cb6g\" (UID: \"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796\") " pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.458410 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.458395 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52g85\" (UniqueName: \"kubernetes.io/projected/e014b55a-8ba8-4eb7-9be2-2d7145e7c965-kube-api-access-52g85\") pod \"ovnkube-node-vkzgb\" (UID: \"e014b55a-8ba8-4eb7-9be2-2d7145e7c965\") " pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.458624 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.458601 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xwn7\" (UniqueName: \"kubernetes.io/projected/8821d59a-1e1c-43d8-80d8-3513deb7478b-kube-api-access-7xwn7\") pod \"node-ca-qkhdg\" (UID: \"8821d59a-1e1c-43d8-80d8-3513deb7478b\") " pod="openshift-image-registry/node-ca-qkhdg" Jul 07 18:49:50.540449 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540422 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d6gx\" (UniqueName: \"kubernetes.io/projected/2f1fd794-cf7b-4072-8df4-1f951b7f2f4c-kube-api-access-5d6gx\") pod \"aws-ebs-csi-driver-node-zmzfh\" (UID: \"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.540449 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540459 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-multus-cni-dir\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.540651 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540504 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d25dd7ba-d534-4f05-b29c-8162b009dfff-etc-tuned\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.540651 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540551 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d25dd7ba-d534-4f05-b29c-8162b009dfff-tmp\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.540651 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540570 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2f1fd794-cf7b-4072-8df4-1f951b7f2f4c-sys-fs\") pod \"aws-ebs-csi-driver-node-zmzfh\" (UID: \"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.540651 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540588 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4be6648f-5fd4-4351-b06b-cee32b5f028f-iptables-alerter-script\") pod \"iptables-alerter-wlrjs\" (UID: \"4be6648f-5fd4-4351-b06b-cee32b5f028f\") " pod="openshift-network-operator/iptables-alerter-wlrjs" Jul 07 18:49:50.540651 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540603 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-etc-systemd\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.540651 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540618 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-lib-modules\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.540651 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540632 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkk7r\" (UniqueName: \"kubernetes.io/projected/7dcc43c9-3758-47a3-aa8d-278166547dad-kube-api-access-lkk7r\") pod \"node-resolver-8bhnt\" (UID: \"7dcc43c9-3758-47a3-aa8d-278166547dad\") " pod="openshift-dns/node-resolver-8bhnt" Jul 07 18:49:50.540868 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540655 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-host-var-lib-cni-multus\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.540868 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540686 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-etc-sysconfig\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.540868 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540700 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-etc-kubernetes\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.540868 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540713 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6tlr\" (UniqueName: \"kubernetes.io/projected/d25dd7ba-d534-4f05-b29c-8162b009dfff-kube-api-access-g6tlr\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.540868 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540728 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2f1fd794-cf7b-4072-8df4-1f951b7f2f4c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zmzfh\" (UID: \"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.540868 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540742 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/570bed30-5e16-475c-9cc4-4e5af148c2e5-kubelet-config\") pod \"global-pull-secret-syncer-ffm5c\" (UID: \"570bed30-5e16-475c-9cc4-4e5af148c2e5\") " pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:49:50.540868 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540755 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-etc-kubernetes\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.540868 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540751 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-multus-cni-dir\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.540868 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540770 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-var-lib-kubelet\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.540868 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540802 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2f1fd794-cf7b-4072-8df4-1f951b7f2f4c-socket-dir\") pod \"aws-ebs-csi-driver-node-zmzfh\" (UID: \"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.540868 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540814 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-var-lib-kubelet\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.540868 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540826 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-cnibin\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.540868 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540855 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dcc43c9-3758-47a3-aa8d-278166547dad-tmp-dir\") pod \"node-resolver-8bhnt\" (UID: \"7dcc43c9-3758-47a3-aa8d-278166547dad\") " pod="openshift-dns/node-resolver-8bhnt" Jul 07 18:49:50.541333 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540878 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-os-release\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.541333 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540900 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-hostroot\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.541333 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540922 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-multus-daemon-config\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.541333 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540948 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs\") pod \"network-metrics-daemon-ccjd5\" (UID: \"536dbfa9-2261-4932-a922-e11a2a764168\") " pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:49:50.541333 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.540972 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5lt5\" (UniqueName: \"kubernetes.io/projected/4be6648f-5fd4-4351-b06b-cee32b5f028f-kube-api-access-s5lt5\") pod \"iptables-alerter-wlrjs\" (UID: \"4be6648f-5fd4-4351-b06b-cee32b5f028f\") " pod="openshift-network-operator/iptables-alerter-wlrjs" Jul 07 18:49:50.541333 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541005 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/570bed30-5e16-475c-9cc4-4e5af148c2e5-dbus\") pod \"global-pull-secret-syncer-ffm5c\" (UID: \"570bed30-5e16-475c-9cc4-4e5af148c2e5\") " pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:49:50.541333 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541028 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-host-run-multus-certs\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.541333 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541053 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lfqs\" (UniqueName: \"kubernetes.io/projected/536dbfa9-2261-4932-a922-e11a2a764168-kube-api-access-2lfqs\") pod \"network-metrics-daemon-ccjd5\" (UID: \"536dbfa9-2261-4932-a922-e11a2a764168\") " pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:49:50.541333 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541075 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-run\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.541333 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541098 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret\") pod \"global-pull-secret-syncer-ffm5c\" (UID: \"570bed30-5e16-475c-9cc4-4e5af148c2e5\") " pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:49:50.541333 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541120 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-system-cni-dir\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.541333 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541143 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-etc-sysctl-d\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.541333 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541164 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-host\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.541333 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541175 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-host-var-lib-cni-multus\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.541333 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541241 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-etc-systemd\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.541985 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541249 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-host\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.541985 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541322 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2f1fd794-cf7b-4072-8df4-1f951b7f2f4c-sys-fs\") pod \"aws-ebs-csi-driver-node-zmzfh\" (UID: \"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.541985 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541340 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-lib-modules\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.541985 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:50.541409 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:49:50.541985 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:50.541490 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs podName:536dbfa9-2261-4932-a922-e11a2a764168 nodeName:}" failed. No retries permitted until 2026-07-07 18:49:51.041468966 +0000 UTC m=+3.011104982 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs") pod "network-metrics-daemon-ccjd5" (UID: "536dbfa9-2261-4932-a922-e11a2a764168") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:49:50.541985 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541674 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2f1fd794-cf7b-4072-8df4-1f951b7f2f4c-socket-dir\") pod \"aws-ebs-csi-driver-node-zmzfh\" (UID: \"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.541985 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541679 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-etc-sysconfig\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.541985 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541733 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-etc-kubernetes\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.541985 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541744 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-cnibin\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.541985 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541744 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4be6648f-5fd4-4351-b06b-cee32b5f028f-iptables-alerter-script\") pod \"iptables-alerter-wlrjs\" (UID: \"4be6648f-5fd4-4351-b06b-cee32b5f028f\") " pod="openshift-network-operator/iptables-alerter-wlrjs" Jul 07 18:49:50.541985 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541761 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bh4z\" (UniqueName: \"kubernetes.io/projected/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-kube-api-access-5bh4z\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.541985 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541772 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/570bed30-5e16-475c-9cc4-4e5af148c2e5-dbus\") pod \"global-pull-secret-syncer-ffm5c\" (UID: \"570bed30-5e16-475c-9cc4-4e5af148c2e5\") " pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:49:50.541985 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:50.541814 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:49:50.541985 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541826 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-system-cni-dir\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.542661 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:50.541863 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret podName:570bed30-5e16-475c-9cc4-4e5af148c2e5 nodeName:}" failed. No retries permitted until 2026-07-07 18:49:51.04184582 +0000 UTC m=+3.011481815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret") pod "global-pull-secret-syncer-ffm5c" (UID: "570bed30-5e16-475c-9cc4-4e5af148c2e5") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:49:50.542661 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541866 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-etc-sysctl-d\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.542661 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541893 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-hostroot\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.542661 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.541915 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-host-run-multus-certs\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.542661 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542024 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-run\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.542661 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542048 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-os-release\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.542661 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542084 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/570bed30-5e16-475c-9cc4-4e5af148c2e5-kubelet-config\") pod \"global-pull-secret-syncer-ffm5c\" (UID: \"570bed30-5e16-475c-9cc4-4e5af148c2e5\") " pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:49:50.542661 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542098 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-etc-kubernetes\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.542661 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542132 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2f1fd794-cf7b-4072-8df4-1f951b7f2f4c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zmzfh\" (UID: \"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.542661 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542105 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7dcc43c9-3758-47a3-aa8d-278166547dad-tmp-dir\") pod \"node-resolver-8bhnt\" (UID: \"7dcc43c9-3758-47a3-aa8d-278166547dad\") " pod="openshift-dns/node-resolver-8bhnt" Jul 07 18:49:50.542661 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542174 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-etc-modprobe-d\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.542661 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542224 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-etc-sysctl-conf\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.542661 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542251 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2f1fd794-cf7b-4072-8df4-1f951b7f2f4c-device-dir\") pod \"aws-ebs-csi-driver-node-zmzfh\" (UID: \"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.542661 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542272 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-host-run-netns\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.543181 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542297 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7dcc43c9-3758-47a3-aa8d-278166547dad-hosts-file\") pod \"node-resolver-8bhnt\" (UID: \"7dcc43c9-3758-47a3-aa8d-278166547dad\") " pod="openshift-dns/node-resolver-8bhnt" Jul 07 18:49:50.543181 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542300 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-etc-modprobe-d\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.543181 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542313 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-multus-socket-dir-parent\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.543181 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542342 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52v7l\" (UniqueName: \"kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l\") pod \"network-check-target-mkg6r\" (UID: \"88340235-0f7e-4e90-bf52-86b3f04aa9ff\") " pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:49:50.543181 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542366 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-host-run-netns\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.543181 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542372 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-sys\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.543181 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542375 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2f1fd794-cf7b-4072-8df4-1f951b7f2f4c-device-dir\") pod \"aws-ebs-csi-driver-node-zmzfh\" (UID: \"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.543181 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542381 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-etc-sysctl-conf\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.543181 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542394 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7dcc43c9-3758-47a3-aa8d-278166547dad-hosts-file\") pod \"node-resolver-8bhnt\" (UID: \"7dcc43c9-3758-47a3-aa8d-278166547dad\") " pod="openshift-dns/node-resolver-8bhnt" Jul 07 18:49:50.543181 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542347 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-multus-socket-dir-parent\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.543181 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542413 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-host-run-k8s-cni-cncf-io\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.543181 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542429 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d25dd7ba-d534-4f05-b29c-8162b009dfff-sys\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.543181 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542432 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-multus-daemon-config\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.543181 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542466 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-host-var-lib-cni-bin\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.543181 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542495 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-host-var-lib-cni-bin\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.543808 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542515 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2f1fd794-cf7b-4072-8df4-1f951b7f2f4c-registration-dir\") pod \"aws-ebs-csi-driver-node-zmzfh\" (UID: \"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.543808 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542536 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-host-run-k8s-cni-cncf-io\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.543808 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542552 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-host-var-lib-kubelet\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.543808 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542577 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-multus-conf-dir\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.543808 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542588 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2f1fd794-cf7b-4072-8df4-1f951b7f2f4c-registration-dir\") pod \"aws-ebs-csi-driver-node-zmzfh\" (UID: \"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.543808 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542602 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-host-var-lib-kubelet\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.543808 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542603 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4be6648f-5fd4-4351-b06b-cee32b5f028f-host-slash\") pod \"iptables-alerter-wlrjs\" (UID: \"4be6648f-5fd4-4351-b06b-cee32b5f028f\") " pod="openshift-network-operator/iptables-alerter-wlrjs" Jul 07 18:49:50.543808 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542636 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4be6648f-5fd4-4351-b06b-cee32b5f028f-host-slash\") pod \"iptables-alerter-wlrjs\" (UID: \"4be6648f-5fd4-4351-b06b-cee32b5f028f\") " pod="openshift-network-operator/iptables-alerter-wlrjs" Jul 07 18:49:50.543808 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542644 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2f1fd794-cf7b-4072-8df4-1f951b7f2f4c-etc-selinux\") pod \"aws-ebs-csi-driver-node-zmzfh\" (UID: \"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.543808 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542637 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-multus-conf-dir\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.543808 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542670 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-cni-binary-copy\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.543808 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.542777 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2f1fd794-cf7b-4072-8df4-1f951b7f2f4c-etc-selinux\") pod \"aws-ebs-csi-driver-node-zmzfh\" (UID: \"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.543808 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.543122 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-cni-binary-copy\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.543808 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.543366 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d25dd7ba-d534-4f05-b29c-8162b009dfff-tmp\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.544768 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.544745 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d25dd7ba-d534-4f05-b29c-8162b009dfff-etc-tuned\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.550995 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:50.550976 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:49:50.551085 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:50.551003 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:49:50.551085 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:50.551016 2564 projected.go:196] Error preparing data for projected volume kube-api-access-52v7l for pod openshift-network-diagnostics/network-check-target-mkg6r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:49:50.551085 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:50.551072 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l podName:88340235-0f7e-4e90-bf52-86b3f04aa9ff nodeName:}" failed. No retries permitted until 2026-07-07 18:49:51.051057859 +0000 UTC m=+3.020693858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-52v7l" (UniqueName: "kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l") pod "network-check-target-mkg6r" (UID: "88340235-0f7e-4e90-bf52-86b3f04aa9ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:49:50.552394 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.552371 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6tlr\" (UniqueName: \"kubernetes.io/projected/d25dd7ba-d534-4f05-b29c-8162b009dfff-kube-api-access-g6tlr\") pod \"tuned-qtjjp\" (UID: \"d25dd7ba-d534-4f05-b29c-8162b009dfff\") " pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.553157 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.553112 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5lt5\" (UniqueName: \"kubernetes.io/projected/4be6648f-5fd4-4351-b06b-cee32b5f028f-kube-api-access-s5lt5\") pod \"iptables-alerter-wlrjs\" (UID: \"4be6648f-5fd4-4351-b06b-cee32b5f028f\") " pod="openshift-network-operator/iptables-alerter-wlrjs" Jul 07 18:49:50.554064 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.554018 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d6gx\" (UniqueName: \"kubernetes.io/projected/2f1fd794-cf7b-4072-8df4-1f951b7f2f4c-kube-api-access-5d6gx\") pod \"aws-ebs-csi-driver-node-zmzfh\" (UID: \"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.555128 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.555106 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bh4z\" (UniqueName: \"kubernetes.io/projected/0d95ab7d-5d1d-4026-b705-72ea50c1ad51-kube-api-access-5bh4z\") pod \"multus-kk8zs\" (UID: \"0d95ab7d-5d1d-4026-b705-72ea50c1ad51\") " pod="openshift-multus/multus-kk8zs" Jul 07 18:49:50.555128 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.555125 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkk7r\" (UniqueName: \"kubernetes.io/projected/7dcc43c9-3758-47a3-aa8d-278166547dad-kube-api-access-lkk7r\") pod \"node-resolver-8bhnt\" (UID: \"7dcc43c9-3758-47a3-aa8d-278166547dad\") " pod="openshift-dns/node-resolver-8bhnt" Jul 07 18:49:50.555955 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.555907 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lfqs\" (UniqueName: \"kubernetes.io/projected/536dbfa9-2261-4932-a922-e11a2a764168-kube-api-access-2lfqs\") pod \"network-metrics-daemon-ccjd5\" (UID: \"536dbfa9-2261-4932-a922-e11a2a764168\") " pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:49:50.570659 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.570617 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-81.ec2.internal" event={"ID":"ee8c04a12a35cab5864ba50e62332606","Type":"ContainerStarted","Data":"e74fb38807691701da6e6a5649ad2c24cb6a51ca16456924f228bece8363d5bf"} Jul 07 18:49:50.571506 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.571489 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-81.ec2.internal" event={"ID":"c7bff29c072ef9b7b9d4f7fce9f2d5cc","Type":"ContainerStarted","Data":"60e1d56b1a9581000f6e77cf900fd5fedaeb9abc018f814ef5426c700184fd32"} Jul 07 18:49:50.633382 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.633357 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w4rcb" Jul 07 18:49:50.639890 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.639645 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:49:50.647295 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.647244 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qkhdg" Jul 07 18:49:50.653527 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.653508 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6cb6g" Jul 07 18:49:50.661134 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.661115 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wlrjs" Jul 07 18:49:50.667760 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.667741 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" Jul 07 18:49:50.675361 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.675339 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" Jul 07 18:49:50.681063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.681043 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 18:49:50.682157 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.682141 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8bhnt" Jul 07 18:49:50.688817 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:50.688794 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kk8zs" Jul 07 18:49:51.045363 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:51.045330 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs\") pod \"network-metrics-daemon-ccjd5\" (UID: \"536dbfa9-2261-4932-a922-e11a2a764168\") " pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:49:51.045519 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:51.045369 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret\") pod \"global-pull-secret-syncer-ffm5c\" (UID: \"570bed30-5e16-475c-9cc4-4e5af148c2e5\") " pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:49:51.045571 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:51.045525 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:49:51.045622 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:51.045600 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:49:51.045668 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:51.045610 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs podName:536dbfa9-2261-4932-a922-e11a2a764168 nodeName:}" failed. No retries permitted until 2026-07-07 18:49:52.045589509 +0000 UTC m=+4.015225508 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs") pod "network-metrics-daemon-ccjd5" (UID: "536dbfa9-2261-4932-a922-e11a2a764168") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:49:51.045713 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:51.045680 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret podName:570bed30-5e16-475c-9cc4-4e5af148c2e5 nodeName:}" failed. No retries permitted until 2026-07-07 18:49:52.04566147 +0000 UTC m=+4.015297478 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret") pod "global-pull-secret-syncer-ffm5c" (UID: "570bed30-5e16-475c-9cc4-4e5af148c2e5") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:49:51.146646 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:51.146612 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52v7l\" (UniqueName: \"kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l\") pod \"network-check-target-mkg6r\" (UID: \"88340235-0f7e-4e90-bf52-86b3f04aa9ff\") " pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:49:51.146811 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:51.146765 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:49:51.146811 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:51.146782 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:49:51.146811 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:51.146794 2564 projected.go:196] Error preparing data for projected volume kube-api-access-52v7l for pod openshift-network-diagnostics/network-check-target-mkg6r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:49:51.146984 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:51.146864 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l podName:88340235-0f7e-4e90-bf52-86b3f04aa9ff nodeName:}" failed. No retries permitted until 2026-07-07 18:49:52.146844573 +0000 UTC m=+4.116480581 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-52v7l" (UniqueName: "kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l") pod "network-check-target-mkg6r" (UID: "88340235-0f7e-4e90-bf52-86b3f04aa9ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:49:51.457719 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:51.457674 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 18:44:50 +0000 UTC" deadline="2028-03-14 14:10:43.142031818 +0000 UTC" Jul 07 18:49:51.457719 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:51.457709 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14779h20m51.684327889s" Jul 07 18:49:51.567314 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:51.567291 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:49:51.567469 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:51.567310 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:49:51.567469 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:51.567400 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mkg6r" podUID="88340235-0f7e-4e90-bf52-86b3f04aa9ff" Jul 07 18:49:51.567564 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:51.567483 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffm5c" podUID="570bed30-5e16-475c-9cc4-4e5af148c2e5" Jul 07 18:49:51.585360 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:51.585333 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod330e5b54_ff72_4853_b885_9a3327275f99.slice/crio-7ba0319458113d33039fe36dbaf19af43d94095ee7c1d79bf24a1c188974a5c1 WatchSource:0}: Error finding container 7ba0319458113d33039fe36dbaf19af43d94095ee7c1d79bf24a1c188974a5c1: Status 404 returned error can't find the container with id 7ba0319458113d33039fe36dbaf19af43d94095ee7c1d79bf24a1c188974a5c1 Jul 07 18:49:51.635511 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:51.635485 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode014b55a_8ba8_4eb7_9be2_2d7145e7c965.slice/crio-d5b8d30caca843d938a209d14a5015ac8d38e25c16d6681a20585a88d6c15321 WatchSource:0}: Error finding container d5b8d30caca843d938a209d14a5015ac8d38e25c16d6681a20585a88d6c15321: Status 404 returned error can't find the container with id d5b8d30caca843d938a209d14a5015ac8d38e25c16d6681a20585a88d6c15321 Jul 07 18:49:51.636675 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:51.636652 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4be6648f_5fd4_4351_b06b_cee32b5f028f.slice/crio-a677e9f0f8d82afcccd4b37c953ee5f0bc8f693179b0f87639d7901f1da36401 WatchSource:0}: Error finding container a677e9f0f8d82afcccd4b37c953ee5f0bc8f693179b0f87639d7901f1da36401: Status 404 returned error can't find the container with id a677e9f0f8d82afcccd4b37c953ee5f0bc8f693179b0f87639d7901f1da36401 Jul 07 18:49:51.637804 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:51.637505 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d95ab7d_5d1d_4026_b705_72ea50c1ad51.slice/crio-d8ac5799321347cb33b11d0b59d8b871f995d177fe79b6f74ddd8ad90895c9ee WatchSource:0}: Error finding container d8ac5799321347cb33b11d0b59d8b871f995d177fe79b6f74ddd8ad90895c9ee: Status 404 returned error can't find the container with id d8ac5799321347cb33b11d0b59d8b871f995d177fe79b6f74ddd8ad90895c9ee Jul 07 18:49:51.638641 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:51.638611 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dcc43c9_3758_47a3_aa8d_278166547dad.slice/crio-693211c9ed0303ecd25b0fd7c07dca00035157b31676515f012e40505c5b65be WatchSource:0}: Error finding container 693211c9ed0303ecd25b0fd7c07dca00035157b31676515f012e40505c5b65be: Status 404 returned error can't find the container with id 693211c9ed0303ecd25b0fd7c07dca00035157b31676515f012e40505c5b65be Jul 07 18:49:51.639672 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:51.639585 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd25dd7ba_d534_4f05_b29c_8162b009dfff.slice/crio-5f5e8d44729392e1756432733b08a377cd4205a7463453c047707662b6fd3efa WatchSource:0}: Error finding container 5f5e8d44729392e1756432733b08a377cd4205a7463453c047707662b6fd3efa: Status 404 returned error can't find the container with id 5f5e8d44729392e1756432733b08a377cd4205a7463453c047707662b6fd3efa Jul 07 18:49:51.640634 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:51.640608 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8821d59a_1e1c_43d8_80d8_3513deb7478b.slice/crio-1f8d5e671c97cf678c0d03b1185d776b242b1d182dba8302bfb2d183439fedb9 WatchSource:0}: Error finding container 1f8d5e671c97cf678c0d03b1185d776b242b1d182dba8302bfb2d183439fedb9: Status 404 returned error can't find the container with id 1f8d5e671c97cf678c0d03b1185d776b242b1d182dba8302bfb2d183439fedb9 Jul 07 18:49:51.641547 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:51.641485 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f1fd794_cf7b_4072_8df4_1f951b7f2f4c.slice/crio-a75b59cd03dffee3bb215b04d3acc5d6857cf8b09ed4417fbf9feb542b4277fd WatchSource:0}: Error finding container a75b59cd03dffee3bb215b04d3acc5d6857cf8b09ed4417fbf9feb542b4277fd: Status 404 returned error can't find the container with id a75b59cd03dffee3bb215b04d3acc5d6857cf8b09ed4417fbf9feb542b4277fd Jul 07 18:49:51.644401 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:49:51.643560 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8a9c668_2d9f_4c07_9b0e_e9e57c20a796.slice/crio-cc5312ce1c74e5803d04b0fe8b312ea45c1bce443a8bf8d25b9a1ad4f000c03d WatchSource:0}: Error finding container cc5312ce1c74e5803d04b0fe8b312ea45c1bce443a8bf8d25b9a1ad4f000c03d: Status 404 returned error can't find the container with id cc5312ce1c74e5803d04b0fe8b312ea45c1bce443a8bf8d25b9a1ad4f000c03d Jul 07 18:49:52.053605 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:52.053418 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs\") pod \"network-metrics-daemon-ccjd5\" (UID: \"536dbfa9-2261-4932-a922-e11a2a764168\") " pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:49:52.053605 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:52.053569 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:49:52.053782 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:52.053625 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs podName:536dbfa9-2261-4932-a922-e11a2a764168 nodeName:}" failed. No retries permitted until 2026-07-07 18:49:54.053609675 +0000 UTC m=+6.023245674 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs") pod "network-metrics-daemon-ccjd5" (UID: "536dbfa9-2261-4932-a922-e11a2a764168") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:49:52.053782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:52.053577 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret\") pod \"global-pull-secret-syncer-ffm5c\" (UID: \"570bed30-5e16-475c-9cc4-4e5af148c2e5\") " pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:49:52.053782 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:52.053644 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:49:52.053782 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:52.053736 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret podName:570bed30-5e16-475c-9cc4-4e5af148c2e5 nodeName:}" failed. No retries permitted until 2026-07-07 18:49:54.053717815 +0000 UTC m=+6.023353829 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret") pod "global-pull-secret-syncer-ffm5c" (UID: "570bed30-5e16-475c-9cc4-4e5af148c2e5") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:49:52.155237 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:52.154593 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52v7l\" (UniqueName: \"kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l\") pod \"network-check-target-mkg6r\" (UID: \"88340235-0f7e-4e90-bf52-86b3f04aa9ff\") " pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:49:52.155237 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:52.154794 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:49:52.155237 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:52.154814 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:49:52.155237 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:52.154826 2564 projected.go:196] Error preparing data for projected volume kube-api-access-52v7l for pod openshift-network-diagnostics/network-check-target-mkg6r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:49:52.155237 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:52.154896 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l podName:88340235-0f7e-4e90-bf52-86b3f04aa9ff nodeName:}" failed. No retries permitted until 2026-07-07 18:49:54.154866636 +0000 UTC m=+6.124502634 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-52v7l" (UniqueName: "kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l") pod "network-check-target-mkg6r" (UID: "88340235-0f7e-4e90-bf52-86b3f04aa9ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:49:52.458367 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:52.458315 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 18:44:50 +0000 UTC" deadline="2028-02-22 11:03:30.374780012 +0000 UTC" Jul 07 18:49:52.458367 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:52.458355 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14272h13m37.916428892s" Jul 07 18:49:52.568833 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:52.568314 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:49:52.568833 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:52.568458 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjd5" podUID="536dbfa9-2261-4932-a922-e11a2a764168" Jul 07 18:49:52.597162 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:52.597086 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wlrjs" event={"ID":"4be6648f-5fd4-4351-b06b-cee32b5f028f","Type":"ContainerStarted","Data":"a677e9f0f8d82afcccd4b37c953ee5f0bc8f693179b0f87639d7901f1da36401"} Jul 07 18:49:52.611129 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:52.611100 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w4rcb" event={"ID":"330e5b54-ff72-4853-b885-9a3327275f99","Type":"ContainerStarted","Data":"7ba0319458113d33039fe36dbaf19af43d94095ee7c1d79bf24a1c188974a5c1"} Jul 07 18:49:52.621174 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:52.621134 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-81.ec2.internal" event={"ID":"c7bff29c072ef9b7b9d4f7fce9f2d5cc","Type":"ContainerStarted","Data":"7a0829c283b72192137f9760b489a7326ab34ff93129e5e9211152c31196b613"} Jul 07 18:49:52.633874 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:52.633847 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6cb6g" event={"ID":"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796","Type":"ContainerStarted","Data":"cc5312ce1c74e5803d04b0fe8b312ea45c1bce443a8bf8d25b9a1ad4f000c03d"} Jul 07 18:49:52.641435 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:52.641362 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" event={"ID":"d25dd7ba-d534-4f05-b29c-8162b009dfff","Type":"ContainerStarted","Data":"5f5e8d44729392e1756432733b08a377cd4205a7463453c047707662b6fd3efa"} Jul 07 18:49:52.642916 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:52.642865 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kk8zs" event={"ID":"0d95ab7d-5d1d-4026-b705-72ea50c1ad51","Type":"ContainerStarted","Data":"d8ac5799321347cb33b11d0b59d8b871f995d177fe79b6f74ddd8ad90895c9ee"} Jul 07 18:49:52.660269 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:52.660241 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" event={"ID":"e014b55a-8ba8-4eb7-9be2-2d7145e7c965","Type":"ContainerStarted","Data":"d5b8d30caca843d938a209d14a5015ac8d38e25c16d6681a20585a88d6c15321"} Jul 07 18:49:52.672430 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:52.671659 2564 generic.go:356] "Generic (PLEG): container finished" podID="ee8c04a12a35cab5864ba50e62332606" containerID="1ac26cb3ce78eff41e44fd9c3d7cb3444ad2994053f20ee26071c4fd179c2fae" exitCode=0 Jul 07 18:49:52.672430 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:52.671727 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-81.ec2.internal" event={"ID":"ee8c04a12a35cab5864ba50e62332606","Type":"ContainerDied","Data":"1ac26cb3ce78eff41e44fd9c3d7cb3444ad2994053f20ee26071c4fd179c2fae"} Jul 07 18:49:52.690273 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:52.690244 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" event={"ID":"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c","Type":"ContainerStarted","Data":"a75b59cd03dffee3bb215b04d3acc5d6857cf8b09ed4417fbf9feb542b4277fd"} Jul 07 18:49:52.691644 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:52.691600 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-81.ec2.internal" podStartSLOduration=3.691584023 podStartE2EDuration="3.691584023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:49:52.638914336 +0000 UTC m=+4.608550354" watchObservedRunningTime="2026-07-07 18:49:52.691584023 +0000 UTC m=+4.661220040" Jul 07 18:49:52.700905 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:52.700870 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8bhnt" event={"ID":"7dcc43c9-3758-47a3-aa8d-278166547dad","Type":"ContainerStarted","Data":"693211c9ed0303ecd25b0fd7c07dca00035157b31676515f012e40505c5b65be"} Jul 07 18:49:52.719450 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:52.719395 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qkhdg" event={"ID":"8821d59a-1e1c-43d8-80d8-3513deb7478b","Type":"ContainerStarted","Data":"1f8d5e671c97cf678c0d03b1185d776b242b1d182dba8302bfb2d183439fedb9"} Jul 07 18:49:53.567047 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:53.567019 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:49:53.567525 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:53.567137 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffm5c" podUID="570bed30-5e16-475c-9cc4-4e5af148c2e5" Jul 07 18:49:53.567525 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:53.567445 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:49:53.567654 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:53.567525 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mkg6r" podUID="88340235-0f7e-4e90-bf52-86b3f04aa9ff" Jul 07 18:49:53.729943 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:53.729857 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-81.ec2.internal" event={"ID":"ee8c04a12a35cab5864ba50e62332606","Type":"ContainerStarted","Data":"7011ed827e12746e7f79eb3240b0f54745a6838650f1111b896954caeb457f50"} Jul 07 18:49:54.073311 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:54.072585 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs\") pod \"network-metrics-daemon-ccjd5\" (UID: \"536dbfa9-2261-4932-a922-e11a2a764168\") " pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:49:54.073311 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:54.072634 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret\") pod \"global-pull-secret-syncer-ffm5c\" (UID: \"570bed30-5e16-475c-9cc4-4e5af148c2e5\") " pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:49:54.073311 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:54.072775 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:49:54.073311 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:54.072833 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret podName:570bed30-5e16-475c-9cc4-4e5af148c2e5 nodeName:}" failed. No retries permitted until 2026-07-07 18:49:58.072816331 +0000 UTC m=+10.042452342 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret") pod "global-pull-secret-syncer-ffm5c" (UID: "570bed30-5e16-475c-9cc4-4e5af148c2e5") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:49:54.073311 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:54.073211 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:49:54.073311 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:54.073256 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs podName:536dbfa9-2261-4932-a922-e11a2a764168 nodeName:}" failed. No retries permitted until 2026-07-07 18:49:58.073242422 +0000 UTC m=+10.042878419 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs") pod "network-metrics-daemon-ccjd5" (UID: "536dbfa9-2261-4932-a922-e11a2a764168") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:49:54.173529 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:54.173494 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52v7l\" (UniqueName: \"kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l\") pod \"network-check-target-mkg6r\" (UID: \"88340235-0f7e-4e90-bf52-86b3f04aa9ff\") " pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:49:54.173724 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:54.173706 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:49:54.173790 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:54.173735 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:49:54.173790 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:54.173746 2564 projected.go:196] Error preparing data for projected volume kube-api-access-52v7l for pod openshift-network-diagnostics/network-check-target-mkg6r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:49:54.173885 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:54.173801 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l podName:88340235-0f7e-4e90-bf52-86b3f04aa9ff nodeName:}" failed. No retries permitted until 2026-07-07 18:49:58.173782809 +0000 UTC m=+10.143418817 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-52v7l" (UniqueName: "kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l") pod "network-check-target-mkg6r" (UID: "88340235-0f7e-4e90-bf52-86b3f04aa9ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:49:54.567750 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:54.567388 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:49:54.567750 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:54.567489 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjd5" podUID="536dbfa9-2261-4932-a922-e11a2a764168" Jul 07 18:49:55.567788 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:55.567499 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:49:55.567788 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:55.567528 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:49:55.567788 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:55.567635 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffm5c" podUID="570bed30-5e16-475c-9cc4-4e5af148c2e5" Jul 07 18:49:55.567788 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:55.567751 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mkg6r" podUID="88340235-0f7e-4e90-bf52-86b3f04aa9ff" Jul 07 18:49:56.568429 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:56.567904 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:49:56.568429 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:56.568047 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjd5" podUID="536dbfa9-2261-4932-a922-e11a2a764168" Jul 07 18:49:57.567728 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:57.567695 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:49:57.567867 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:57.567837 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mkg6r" podUID="88340235-0f7e-4e90-bf52-86b3f04aa9ff" Jul 07 18:49:57.567929 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:57.567910 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:49:57.568060 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:57.567983 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffm5c" podUID="570bed30-5e16-475c-9cc4-4e5af148c2e5" Jul 07 18:49:58.104306 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:58.104269 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs\") pod \"network-metrics-daemon-ccjd5\" (UID: \"536dbfa9-2261-4932-a922-e11a2a764168\") " pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:49:58.104752 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:58.104328 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret\") pod \"global-pull-secret-syncer-ffm5c\" (UID: \"570bed30-5e16-475c-9cc4-4e5af148c2e5\") " pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:49:58.104752 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:58.104517 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:49:58.104752 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:58.104580 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret podName:570bed30-5e16-475c-9cc4-4e5af148c2e5 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:06.104560971 +0000 UTC m=+18.074196968 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret") pod "global-pull-secret-syncer-ffm5c" (UID: "570bed30-5e16-475c-9cc4-4e5af148c2e5") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:49:58.104752 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:58.104605 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:49:58.104752 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:58.104644 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs podName:536dbfa9-2261-4932-a922-e11a2a764168 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:06.104632758 +0000 UTC m=+18.074268768 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs") pod "network-metrics-daemon-ccjd5" (UID: "536dbfa9-2261-4932-a922-e11a2a764168") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:49:58.205034 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:58.204998 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52v7l\" (UniqueName: \"kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l\") pod \"network-check-target-mkg6r\" (UID: \"88340235-0f7e-4e90-bf52-86b3f04aa9ff\") " pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:49:58.205277 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:58.205216 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:49:58.205277 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:58.205240 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:49:58.205277 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:58.205253 2564 projected.go:196] Error preparing data for projected volume kube-api-access-52v7l for pod openshift-network-diagnostics/network-check-target-mkg6r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:49:58.205422 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:58.205311 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l podName:88340235-0f7e-4e90-bf52-86b3f04aa9ff nodeName:}" failed. No retries permitted until 2026-07-07 18:50:06.20529339 +0000 UTC m=+18.174929401 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-52v7l" (UniqueName: "kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l") pod "network-check-target-mkg6r" (UID: "88340235-0f7e-4e90-bf52-86b3f04aa9ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:49:58.568767 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:58.568735 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:49:58.568934 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:58.568855 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjd5" podUID="536dbfa9-2261-4932-a922-e11a2a764168" Jul 07 18:49:59.567148 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:59.567115 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:49:59.567669 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:49:59.567130 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:49:59.567669 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:59.567267 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mkg6r" podUID="88340235-0f7e-4e90-bf52-86b3f04aa9ff" Jul 07 18:49:59.567669 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:49:59.567365 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffm5c" podUID="570bed30-5e16-475c-9cc4-4e5af148c2e5" Jul 07 18:50:00.567665 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:00.567634 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:50:00.568100 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:00.567784 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjd5" podUID="536dbfa9-2261-4932-a922-e11a2a764168" Jul 07 18:50:01.567124 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:01.567099 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:50:01.567298 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:01.567102 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:50:01.567298 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:01.567187 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mkg6r" podUID="88340235-0f7e-4e90-bf52-86b3f04aa9ff" Jul 07 18:50:01.567298 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:01.567275 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffm5c" podUID="570bed30-5e16-475c-9cc4-4e5af148c2e5" Jul 07 18:50:02.567889 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:02.567861 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:50:02.568301 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:02.567999 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjd5" podUID="536dbfa9-2261-4932-a922-e11a2a764168" Jul 07 18:50:03.567420 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:03.567385 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:50:03.567589 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:03.567506 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mkg6r" podUID="88340235-0f7e-4e90-bf52-86b3f04aa9ff" Jul 07 18:50:03.567589 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:03.567552 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:50:03.567705 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:03.567645 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffm5c" podUID="570bed30-5e16-475c-9cc4-4e5af148c2e5" Jul 07 18:50:04.567742 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:04.567708 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:50:04.568129 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:04.567860 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjd5" podUID="536dbfa9-2261-4932-a922-e11a2a764168" Jul 07 18:50:05.567799 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:05.567771 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:50:05.567799 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:05.567791 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:50:05.568223 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:05.567905 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffm5c" podUID="570bed30-5e16-475c-9cc4-4e5af148c2e5" Jul 07 18:50:05.568223 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:05.568026 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mkg6r" podUID="88340235-0f7e-4e90-bf52-86b3f04aa9ff" Jul 07 18:50:05.752455 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:05.752272 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6cb6g" event={"ID":"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796","Type":"ContainerStarted","Data":"bd35e406781147ba6fc3fca6a48ea81afab3075f52e3e3998ae2b71ee250e6ee"} Jul 07 18:50:05.753652 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:05.753623 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" event={"ID":"d25dd7ba-d534-4f05-b29c-8162b009dfff","Type":"ContainerStarted","Data":"5d7323e9070cd1094049ce063d30e78a47685510602bf3b6f00b753990908b05"} Jul 07 18:50:05.754932 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:05.754908 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" event={"ID":"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c","Type":"ContainerStarted","Data":"bf9022a862121fa5e2d56114d83737dd61a6c010d26945224a014307b341f680"} Jul 07 18:50:05.756070 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:05.756046 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8bhnt" event={"ID":"7dcc43c9-3758-47a3-aa8d-278166547dad","Type":"ContainerStarted","Data":"353a36e315c0f6cc1946b49aa3962cd934f04739190cab91fb9a8d198025f8a5"} Jul 07 18:50:05.757407 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:05.757383 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qkhdg" event={"ID":"8821d59a-1e1c-43d8-80d8-3513deb7478b","Type":"ContainerStarted","Data":"10daf2cdf4959f8bd06cdb87747efad6e18bbd8d5e8236ca629c6b71bf303368"} Jul 07 18:50:05.758634 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:05.758612 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w4rcb" event={"ID":"330e5b54-ff72-4853-b885-9a3327275f99","Type":"ContainerStarted","Data":"eb5ebcc4b65a9a2001001a19946b26f1d7aa8d3771360636eeb847fac19749c5"} Jul 07 18:50:05.782856 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:05.782820 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-81.ec2.internal" podStartSLOduration=16.782805437 podStartE2EDuration="16.782805437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:49:53.749186748 +0000 UTC m=+5.718822781" watchObservedRunningTime="2026-07-07 18:50:05.782805437 +0000 UTC m=+17.752441453" Jul 07 18:50:05.802634 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:05.802583 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qtjjp" podStartSLOduration=4.058156189 podStartE2EDuration="17.802570152s" totalImagesPullingTime="13.744413963s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:05.802546552 +0000 UTC m=+17.772182578" watchObservedRunningTime="2026-07-07 18:50:05.802570152 +0000 UTC m=+17.772206168" Jul 07 18:50:05.822458 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:05.822421 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-w4rcb" podStartSLOduration=4.041591464 podStartE2EDuration="17.822411704s" totalImagesPullingTime="13.78082024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:49:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:05.822253447 +0000 UTC m=+17.791889480" watchObservedRunningTime="2026-07-07 18:50:05.822411704 +0000 UTC m=+17.792047772" Jul 07 18:50:05.843821 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:05.843789 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8bhnt" podStartSLOduration=4.100263648 podStartE2EDuration="17.843777007s" totalImagesPullingTime="13.743513359s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:05.843416026 +0000 UTC m=+17.813052060" watchObservedRunningTime="2026-07-07 18:50:05.843777007 +0000 UTC m=+17.813413022" Jul 07 18:50:05.862920 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:05.862878 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qkhdg" podStartSLOduration=4.121080771 podStartE2EDuration="17.862864843s" totalImagesPullingTime="13.741784072s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:05.862352795 +0000 UTC m=+17.831988806" watchObservedRunningTime="2026-07-07 18:50:05.862864843 +0000 UTC m=+17.832500951" Jul 07 18:50:06.164857 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:06.164806 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs\") pod \"network-metrics-daemon-ccjd5\" (UID: \"536dbfa9-2261-4932-a922-e11a2a764168\") " pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:50:06.164857 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:06.164837 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret\") pod \"global-pull-secret-syncer-ffm5c\" (UID: \"570bed30-5e16-475c-9cc4-4e5af148c2e5\") " pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:50:06.165009 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:06.164949 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:50:06.165009 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:06.164993 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:50:06.165009 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:06.165007 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs podName:536dbfa9-2261-4932-a922-e11a2a764168 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:22.164992385 +0000 UTC m=+34.134628384 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs") pod "network-metrics-daemon-ccjd5" (UID: "536dbfa9-2261-4932-a922-e11a2a764168") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:50:06.165116 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:06.165031 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret podName:570bed30-5e16-475c-9cc4-4e5af148c2e5 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:22.165020267 +0000 UTC m=+34.134656261 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret") pod "global-pull-secret-syncer-ffm5c" (UID: "570bed30-5e16-475c-9cc4-4e5af148c2e5") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:50:06.265287 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:06.265261 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52v7l\" (UniqueName: \"kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l\") pod \"network-check-target-mkg6r\" (UID: \"88340235-0f7e-4e90-bf52-86b3f04aa9ff\") " pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:50:06.265420 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:06.265403 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:50:06.265456 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:06.265426 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:50:06.265456 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:06.265436 2564 projected.go:196] Error preparing data for projected volume kube-api-access-52v7l for pod openshift-network-diagnostics/network-check-target-mkg6r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:50:06.265510 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:06.265480 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l podName:88340235-0f7e-4e90-bf52-86b3f04aa9ff nodeName:}" failed. No retries permitted until 2026-07-07 18:50:22.265466382 +0000 UTC m=+34.235102376 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-52v7l" (UniqueName: "kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l") pod "network-check-target-mkg6r" (UID: "88340235-0f7e-4e90-bf52-86b3f04aa9ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:50:06.567785 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:06.567755 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:50:06.567928 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:06.567885 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjd5" podUID="536dbfa9-2261-4932-a922-e11a2a764168" Jul 07 18:50:06.761799 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:06.761762 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kk8zs" event={"ID":"0d95ab7d-5d1d-4026-b705-72ea50c1ad51","Type":"ContainerStarted","Data":"baa4b46f40797ccca39fcf40fb503fd10166cace01d1c6bbc1d9980dc98ae191"} Jul 07 18:50:06.763257 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:06.763230 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wlrjs" event={"ID":"4be6648f-5fd4-4351-b06b-cee32b5f028f","Type":"ContainerStarted","Data":"69e34a6090b93eb10db692d3d86833101a97bc416900ea4ec127b58d3313cbb1"} Jul 07 18:50:06.764661 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:06.764629 2564 generic.go:356] "Generic (PLEG): container finished" podID="b8a9c668-2d9f-4c07-9b0e-e9e57c20a796" containerID="bd35e406781147ba6fc3fca6a48ea81afab3075f52e3e3998ae2b71ee250e6ee" exitCode=0 Jul 07 18:50:06.764777 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:06.764747 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6cb6g" event={"ID":"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796","Type":"ContainerDied","Data":"bd35e406781147ba6fc3fca6a48ea81afab3075f52e3e3998ae2b71ee250e6ee"} Jul 07 18:50:06.785845 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:06.785797 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-kk8zs" podStartSLOduration=4.548343941 podStartE2EDuration="18.785784065s" totalImagesPullingTime="14.237440124s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:06.784768268 +0000 UTC m=+18.754404302" watchObservedRunningTime="2026-07-07 18:50:06.785784065 +0000 UTC m=+18.755420088" Jul 07 18:50:06.941727 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:06.941705 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 18:50:06.982204 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:06.982162 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w4rcb_330e5b54-ff72-4853-b885-9a3327275f99/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:07.141971 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:07.141911 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w4rcb_330e5b54-ff72-4853-b885-9a3327275f99/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:07.251499 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:07.251462 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w4rcb_330e5b54-ff72-4853-b885-9a3327275f99/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:07.357882 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:07.357841 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w4rcb_330e5b54-ff72-4853-b885-9a3327275f99/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:07.416477 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:07.416295 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T18:50:06.941722856Z","UUID":"b0f752fa-7c84-4b4f-a46b-7dc84e47e929","Handler":null,"Name":"","Endpoint":""} Jul 07 18:50:07.417959 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:07.417927 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 18:50:07.418058 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:07.417966 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 18:50:07.464301 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:07.464270 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w4rcb_330e5b54-ff72-4853-b885-9a3327275f99/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:07.567155 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:07.567118 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:50:07.567341 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:07.567233 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mkg6r" podUID="88340235-0f7e-4e90-bf52-86b3f04aa9ff" Jul 07 18:50:07.567642 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:07.567622 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:50:07.567778 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:07.567724 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffm5c" podUID="570bed30-5e16-475c-9cc4-4e5af148c2e5" Jul 07 18:50:07.622829 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:07.622797 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w4rcb_330e5b54-ff72-4853-b885-9a3327275f99/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:07.727015 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:07.726724 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w4rcb_330e5b54-ff72-4853-b885-9a3327275f99/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:07.770371 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:07.770340 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" event={"ID":"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c","Type":"ContainerStarted","Data":"af23bfb04e6c5154989dc6cd7bba0629ee4f3de369104568aabd66d08d66b385"} Jul 07 18:50:08.568160 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:08.568131 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:50:08.568335 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:08.568259 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjd5" podUID="536dbfa9-2261-4932-a922-e11a2a764168" Jul 07 18:50:08.773743 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:08.773714 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" event={"ID":"2f1fd794-cf7b-4072-8df4-1f951b7f2f4c","Type":"ContainerStarted","Data":"46875acb84a5f41b60dfab231701d7e6e1e553fc4bbaa35a87f6a7c573eaca5e"} Jul 07 18:50:08.801816 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:08.801760 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wlrjs" podStartSLOduration=7.054140386 podStartE2EDuration="20.801746752s" totalImagesPullingTime="13.747606366s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:06.831462274 +0000 UTC m=+18.801098289" watchObservedRunningTime="2026-07-07 18:50:08.801746752 +0000 UTC m=+20.771382769" Jul 07 18:50:09.564571 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:09.564535 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-w4rcb" Jul 07 18:50:09.565100 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:09.565081 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-w4rcb" Jul 07 18:50:09.567165 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:09.567140 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:50:09.567289 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:09.567263 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mkg6r" podUID="88340235-0f7e-4e90-bf52-86b3f04aa9ff" Jul 07 18:50:09.567384 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:09.567320 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:50:09.567470 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:09.567445 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffm5c" podUID="570bed30-5e16-475c-9cc4-4e5af148c2e5" Jul 07 18:50:09.597778 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:09.597739 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zmzfh" podStartSLOduration=5.959713371 podStartE2EDuration="21.597725468s" totalImagesPullingTime="15.638012097s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:08.801590304 +0000 UTC m=+20.771226343" watchObservedRunningTime="2026-07-07 18:50:09.597725468 +0000 UTC m=+21.567361483" Jul 07 18:50:09.775821 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:09.775784 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-w4rcb" Jul 07 18:50:09.776257 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:09.776243 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-w4rcb" Jul 07 18:50:09.879329 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:09.879227 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w4rcb_330e5b54-ff72-4853-b885-9a3327275f99/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:10.567437 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:10.567406 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:50:10.567621 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:10.567550 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjd5" podUID="536dbfa9-2261-4932-a922-e11a2a764168" Jul 07 18:50:11.567804 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:11.567777 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:50:11.568218 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:11.567881 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffm5c" podUID="570bed30-5e16-475c-9cc4-4e5af148c2e5" Jul 07 18:50:11.568218 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:11.567949 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:50:11.568218 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:11.568052 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mkg6r" podUID="88340235-0f7e-4e90-bf52-86b3f04aa9ff" Jul 07 18:50:12.567865 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:12.567830 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:50:12.568385 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:12.567974 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjd5" podUID="536dbfa9-2261-4932-a922-e11a2a764168" Jul 07 18:50:13.567565 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:13.567536 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:50:13.567743 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:13.567536 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:50:13.567743 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:13.567651 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffm5c" podUID="570bed30-5e16-475c-9cc4-4e5af148c2e5" Jul 07 18:50:13.567909 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:13.567740 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mkg6r" podUID="88340235-0f7e-4e90-bf52-86b3f04aa9ff" Jul 07 18:50:14.567502 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:14.567475 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:50:14.567718 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:14.567573 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjd5" podUID="536dbfa9-2261-4932-a922-e11a2a764168" Jul 07 18:50:14.873380 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:14.873310 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w4rcb_330e5b54-ff72-4853-b885-9a3327275f99/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:15.016396 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:15.016359 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w4rcb_330e5b54-ff72-4853-b885-9a3327275f99/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:15.295893 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:15.295740 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w4rcb_330e5b54-ff72-4853-b885-9a3327275f99/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:15.392059 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:15.392035 2564 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w4rcb_330e5b54-ff72-4853-b885-9a3327275f99/konnectivity-agent/0.log" limit=1024 Jul 07 18:50:15.567991 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:15.567964 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:50:15.568329 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:15.568064 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mkg6r" podUID="88340235-0f7e-4e90-bf52-86b3f04aa9ff" Jul 07 18:50:15.568329 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:15.568118 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:50:15.568329 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:15.568232 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffm5c" podUID="570bed30-5e16-475c-9cc4-4e5af148c2e5" Jul 07 18:50:15.789058 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:15.789026 2564 generic.go:356] "Generic (PLEG): container finished" podID="b8a9c668-2d9f-4c07-9b0e-e9e57c20a796" containerID="8406d65babdb3d5778144993cf02c3d56aab2bfa6cce6d344b0b75f7884c2038" exitCode=0 Jul 07 18:50:15.789302 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:15.789111 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6cb6g" event={"ID":"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796","Type":"ContainerDied","Data":"8406d65babdb3d5778144993cf02c3d56aab2bfa6cce6d344b0b75f7884c2038"} Jul 07 18:50:15.792106 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:15.791987 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" event={"ID":"e014b55a-8ba8-4eb7-9be2-2d7145e7c965","Type":"ContainerStarted","Data":"12abdf84d90f5d11743b5d64b96de695f143dc394bb3128c8eca9eee2f4f1867"} Jul 07 18:50:15.792409 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:15.792386 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" event={"ID":"e014b55a-8ba8-4eb7-9be2-2d7145e7c965","Type":"ContainerStarted","Data":"be5d0bf11c1833ae45a5de53d840cf83fa02fc4702f5d4ba06f1dfb06961f4a7"} Jul 07 18:50:15.793630 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:15.793606 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" event={"ID":"e014b55a-8ba8-4eb7-9be2-2d7145e7c965","Type":"ContainerStarted","Data":"4bf417275fa1c56c42b622db94bed2b10d9c3cf656447ef766346548a25e73c8"} Jul 07 18:50:15.793728 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:15.793686 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" event={"ID":"e014b55a-8ba8-4eb7-9be2-2d7145e7c965","Type":"ContainerStarted","Data":"e741490a622ade32cb356c986c2ec967bd21a0732e9ebc815635c85ef1e05645"} Jul 07 18:50:15.793785 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:15.793746 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" event={"ID":"e014b55a-8ba8-4eb7-9be2-2d7145e7c965","Type":"ContainerStarted","Data":"a6291ab6a87a666ca562fe0d47d557663577201c83542f8dde2b99a8933fedde"} Jul 07 18:50:15.793830 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:15.793812 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" event={"ID":"e014b55a-8ba8-4eb7-9be2-2d7145e7c965","Type":"ContainerStarted","Data":"5189b101d1bac898d15420c5ac162f68743d7d113d6b777ace7265a5a55a878d"} Jul 07 18:50:16.567343 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:16.567312 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:50:16.567919 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:16.567460 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjd5" podUID="536dbfa9-2261-4932-a922-e11a2a764168" Jul 07 18:50:17.567441 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:17.567421 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:50:17.567710 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:17.567421 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:50:17.567710 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:17.567577 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffm5c" podUID="570bed30-5e16-475c-9cc4-4e5af148c2e5" Jul 07 18:50:17.567710 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:17.567506 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mkg6r" podUID="88340235-0f7e-4e90-bf52-86b3f04aa9ff" Jul 07 18:50:17.799458 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:17.799384 2564 generic.go:356] "Generic (PLEG): container finished" podID="b8a9c668-2d9f-4c07-9b0e-e9e57c20a796" containerID="418eab06925d1160b5ce9ae99916787d9b2bad7c27e781a8d6eee452e2e30bbf" exitCode=0 Jul 07 18:50:17.799595 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:17.799472 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6cb6g" event={"ID":"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796","Type":"ContainerDied","Data":"418eab06925d1160b5ce9ae99916787d9b2bad7c27e781a8d6eee452e2e30bbf"} Jul 07 18:50:17.802566 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:17.802543 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" event={"ID":"e014b55a-8ba8-4eb7-9be2-2d7145e7c965","Type":"ContainerStarted","Data":"c72086eb2f6f9f920ee86d249c6cdfaacb4b47d11f3fec45cf50ef683e22ec81"} Jul 07 18:50:18.568574 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:18.568545 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:50:18.569003 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:18.568634 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjd5" podUID="536dbfa9-2261-4932-a922-e11a2a764168" Jul 07 18:50:19.567715 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:19.567692 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:50:19.567715 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:19.567707 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:50:19.567884 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:19.567776 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mkg6r" podUID="88340235-0f7e-4e90-bf52-86b3f04aa9ff" Jul 07 18:50:19.567884 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:19.567831 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffm5c" podUID="570bed30-5e16-475c-9cc4-4e5af148c2e5" Jul 07 18:50:19.809318 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:19.809181 2564 generic.go:356] "Generic (PLEG): container finished" podID="b8a9c668-2d9f-4c07-9b0e-e9e57c20a796" containerID="97e4b530cb686b8952293999eaf4685ff77d8760a614213710be7bdfd6799528" exitCode=0 Jul 07 18:50:19.809686 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:19.809225 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6cb6g" event={"ID":"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796","Type":"ContainerDied","Data":"97e4b530cb686b8952293999eaf4685ff77d8760a614213710be7bdfd6799528"} Jul 07 18:50:19.812971 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:19.812949 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" event={"ID":"e014b55a-8ba8-4eb7-9be2-2d7145e7c965","Type":"ContainerStarted","Data":"50a525dec051c39e60f74c84df5a4a3284f482bbede58d1a527ffb85f55ea6b7"} Jul 07 18:50:19.813181 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:19.813163 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:50:19.813284 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:19.813274 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:50:19.813351 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:19.813326 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:50:19.827556 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:19.827495 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:50:19.827640 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:19.827617 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:50:20.567508 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:20.567478 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:50:20.567651 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:20.567579 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjd5" podUID="536dbfa9-2261-4932-a922-e11a2a764168" Jul 07 18:50:21.568063 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:21.568033 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:50:21.568551 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:21.568070 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:50:21.568551 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:21.568161 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mkg6r" podUID="88340235-0f7e-4e90-bf52-86b3f04aa9ff" Jul 07 18:50:21.568551 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:21.568315 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffm5c" podUID="570bed30-5e16-475c-9cc4-4e5af148c2e5" Jul 07 18:50:21.647677 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:21.647622 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" podStartSLOduration=10.390781236 podStartE2EDuration="33.64760197s" totalImagesPullingTime="23.256820734s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:19.86211771 +0000 UTC m=+31.831753746" watchObservedRunningTime="2026-07-07 18:50:21.64760197 +0000 UTC m=+33.617237988" Jul 07 18:50:21.648368 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:21.648346 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ffm5c"] Jul 07 18:50:21.652170 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:21.652147 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mkg6r"] Jul 07 18:50:21.662282 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:21.662100 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ccjd5"] Jul 07 18:50:21.662282 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:21.662211 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:50:21.662432 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:21.662323 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjd5" podUID="536dbfa9-2261-4932-a922-e11a2a764168" Jul 07 18:50:21.816506 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:21.816478 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:50:21.816645 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:21.816486 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:50:21.816645 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:21.816584 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mkg6r" podUID="88340235-0f7e-4e90-bf52-86b3f04aa9ff" Jul 07 18:50:21.816720 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:21.816647 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffm5c" podUID="570bed30-5e16-475c-9cc4-4e5af148c2e5" Jul 07 18:50:22.184310 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:22.184238 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs\") pod \"network-metrics-daemon-ccjd5\" (UID: \"536dbfa9-2261-4932-a922-e11a2a764168\") " pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:50:22.184310 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:22.184271 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret\") pod \"global-pull-secret-syncer-ffm5c\" (UID: \"570bed30-5e16-475c-9cc4-4e5af148c2e5\") " pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:50:22.184467 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:22.184381 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:50:22.184467 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:22.184437 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs podName:536dbfa9-2261-4932-a922-e11a2a764168 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:54.184422767 +0000 UTC m=+66.154058763 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs") pod "network-metrics-daemon-ccjd5" (UID: "536dbfa9-2261-4932-a922-e11a2a764168") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 18:50:22.184542 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:22.184382 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 18:50:22.184542 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:22.184507 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret podName:570bed30-5e16-475c-9cc4-4e5af148c2e5 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:54.184495571 +0000 UTC m=+66.154131565 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret") pod "global-pull-secret-syncer-ffm5c" (UID: "570bed30-5e16-475c-9cc4-4e5af148c2e5") : object "kube-system"/"original-pull-secret" not registered Jul 07 18:50:22.284798 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:22.284770 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52v7l\" (UniqueName: \"kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l\") pod \"network-check-target-mkg6r\" (UID: \"88340235-0f7e-4e90-bf52-86b3f04aa9ff\") " pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:50:22.284940 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:22.284899 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 18:50:22.284940 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:22.284918 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 18:50:22.284940 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:22.284929 2564 projected.go:196] Error preparing data for projected volume kube-api-access-52v7l for pod openshift-network-diagnostics/network-check-target-mkg6r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:50:22.285111 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:22.284981 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l podName:88340235-0f7e-4e90-bf52-86b3f04aa9ff nodeName:}" failed. No retries permitted until 2026-07-07 18:50:54.284965332 +0000 UTC m=+66.254601337 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-52v7l" (UniqueName: "kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l") pod "network-check-target-mkg6r" (UID: "88340235-0f7e-4e90-bf52-86b3f04aa9ff") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 18:50:23.567733 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:23.567704 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:50:23.567733 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:23.567720 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:50:23.568120 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:23.567704 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:50:23.568120 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:23.567812 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mkg6r" podUID="88340235-0f7e-4e90-bf52-86b3f04aa9ff" Jul 07 18:50:23.568120 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:23.567878 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjd5" podUID="536dbfa9-2261-4932-a922-e11a2a764168" Jul 07 18:50:23.568120 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:23.567991 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffm5c" podUID="570bed30-5e16-475c-9cc4-4e5af148c2e5" Jul 07 18:50:25.567489 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:25.567318 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:50:25.568011 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:25.567350 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:50:25.568011 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:25.567553 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffm5c" podUID="570bed30-5e16-475c-9cc4-4e5af148c2e5" Jul 07 18:50:25.568011 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:25.567651 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mkg6r" podUID="88340235-0f7e-4e90-bf52-86b3f04aa9ff" Jul 07 18:50:25.568011 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:25.567375 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:50:25.568011 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:25.567744 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjd5" podUID="536dbfa9-2261-4932-a922-e11a2a764168" Jul 07 18:50:25.827949 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:25.827872 2564 generic.go:356] "Generic (PLEG): container finished" podID="b8a9c668-2d9f-4c07-9b0e-e9e57c20a796" containerID="393a727b5226896ba09f580c149d970894ccca0bb071131fc03ef8d990e57aaf" exitCode=0 Jul 07 18:50:25.827949 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:25.827908 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6cb6g" event={"ID":"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796","Type":"ContainerDied","Data":"393a727b5226896ba09f580c149d970894ccca0bb071131fc03ef8d990e57aaf"} Jul 07 18:50:26.832415 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:26.832381 2564 generic.go:356] "Generic (PLEG): container finished" podID="b8a9c668-2d9f-4c07-9b0e-e9e57c20a796" containerID="02aaa70870a62110a8d2b702304c9bb00194518e385db53a84e592a90b8d57a6" exitCode=0 Jul 07 18:50:26.832867 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:26.832443 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6cb6g" event={"ID":"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796","Type":"ContainerDied","Data":"02aaa70870a62110a8d2b702304c9bb00194518e385db53a84e592a90b8d57a6"} Jul 07 18:50:27.567109 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:27.567029 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:50:27.567290 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:27.567030 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:50:27.567290 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:27.567134 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ccjd5" podUID="536dbfa9-2261-4932-a922-e11a2a764168" Jul 07 18:50:27.567290 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:27.567030 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:50:27.567290 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:27.567229 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mkg6r" podUID="88340235-0f7e-4e90-bf52-86b3f04aa9ff" Jul 07 18:50:27.567436 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:27.567303 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffm5c" podUID="570bed30-5e16-475c-9cc4-4e5af148c2e5" Jul 07 18:50:27.821402 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:27.821331 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-81.ec2.internal" event="NodeReady" Jul 07 18:50:27.821537 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:27.821426 2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 18:50:27.837937 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:27.837910 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6cb6g" event={"ID":"b8a9c668-2d9f-4c07-9b0e-e9e57c20a796","Type":"ContainerStarted","Data":"93ef7f56fc1a1fbe308c127e3c33b21b67ca3367ca0f58cab9bcdc516ab5093c"} Jul 07 18:50:27.861734 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:27.861692 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6cb6g" podStartSLOduration=10.460657861 podStartE2EDuration="39.861678242s" totalImagesPullingTime="29.401020381s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:48 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:27.861043864 +0000 UTC m=+39.830679881" watchObservedRunningTime="2026-07-07 18:50:27.861678242 +0000 UTC m=+39.831314259" Jul 07 18:50:27.875584 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:27.875558 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-b57q9"] Jul 07 18:50:27.907286 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:27.907262 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bphf7"] Jul 07 18:50:27.907400 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:27.907387 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b57q9" Jul 07 18:50:27.909679 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:27.909656 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 18:50:27.909780 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:27.909688 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-krxdw\"" Jul 07 18:50:27.909954 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:27.909940 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 18:50:27.921209 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:27.921182 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b57q9"] Jul 07 18:50:27.921277 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:27.921230 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bphf7"] Jul 07 18:50:27.921277 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:27.921275 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bphf7" Jul 07 18:50:27.923417 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:27.923392 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 18:50:27.923524 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:27.923490 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 18:50:27.923584 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:27.923522 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9shx6\"" Jul 07 18:50:27.923584 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:27.923538 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 18:50:28.024264 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:28.024226 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-metrics-tls\") pod \"dns-default-b57q9\" (UID: \"fa1690c9-bef6-43d0-9aff-2359f4dfc39f\") " pod="openshift-dns/dns-default-b57q9" Jul 07 18:50:28.024448 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:28.024285 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf6f4b56-2567-469f-a0d4-bc586ab3c6bf-cert\") pod \"ingress-canary-bphf7\" (UID: \"cf6f4b56-2567-469f-a0d4-bc586ab3c6bf\") " pod="openshift-ingress-canary/ingress-canary-bphf7" Jul 07 18:50:28.024448 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:28.024327 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2wnh\" (UniqueName: \"kubernetes.io/projected/cf6f4b56-2567-469f-a0d4-bc586ab3c6bf-kube-api-access-z2wnh\") pod \"ingress-canary-bphf7\" (UID: \"cf6f4b56-2567-469f-a0d4-bc586ab3c6bf\") " pod="openshift-ingress-canary/ingress-canary-bphf7" Jul 07 18:50:28.024448 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:28.024352 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-tmp-dir\") pod \"dns-default-b57q9\" (UID: \"fa1690c9-bef6-43d0-9aff-2359f4dfc39f\") " pod="openshift-dns/dns-default-b57q9" Jul 07 18:50:28.024448 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:28.024377 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfkj9\" (UniqueName: \"kubernetes.io/projected/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-kube-api-access-sfkj9\") pod \"dns-default-b57q9\" (UID: \"fa1690c9-bef6-43d0-9aff-2359f4dfc39f\") " pod="openshift-dns/dns-default-b57q9" Jul 07 18:50:28.024448 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:28.024419 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-config-volume\") pod \"dns-default-b57q9\" (UID: \"fa1690c9-bef6-43d0-9aff-2359f4dfc39f\") " pod="openshift-dns/dns-default-b57q9" Jul 07 18:50:28.125549 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:28.125483 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-metrics-tls\") pod \"dns-default-b57q9\" (UID: \"fa1690c9-bef6-43d0-9aff-2359f4dfc39f\") " pod="openshift-dns/dns-default-b57q9" Jul 07 18:50:28.125549 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:28.125534 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf6f4b56-2567-469f-a0d4-bc586ab3c6bf-cert\") pod \"ingress-canary-bphf7\" (UID: \"cf6f4b56-2567-469f-a0d4-bc586ab3c6bf\") " pod="openshift-ingress-canary/ingress-canary-bphf7" Jul 07 18:50:28.125718 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:28.125571 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2wnh\" (UniqueName: \"kubernetes.io/projected/cf6f4b56-2567-469f-a0d4-bc586ab3c6bf-kube-api-access-z2wnh\") pod \"ingress-canary-bphf7\" (UID: \"cf6f4b56-2567-469f-a0d4-bc586ab3c6bf\") " pod="openshift-ingress-canary/ingress-canary-bphf7" Jul 07 18:50:28.125718 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:28.125593 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-tmp-dir\") pod \"dns-default-b57q9\" (UID: \"fa1690c9-bef6-43d0-9aff-2359f4dfc39f\") " pod="openshift-dns/dns-default-b57q9" Jul 07 18:50:28.125718 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:28.125620 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfkj9\" (UniqueName: \"kubernetes.io/projected/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-kube-api-access-sfkj9\") pod \"dns-default-b57q9\" (UID: \"fa1690c9-bef6-43d0-9aff-2359f4dfc39f\") " pod="openshift-dns/dns-default-b57q9" Jul 07 18:50:28.125718 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:28.125660 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-config-volume\") pod \"dns-default-b57q9\" (UID: \"fa1690c9-bef6-43d0-9aff-2359f4dfc39f\") " pod="openshift-dns/dns-default-b57q9" Jul 07 18:50:28.125718 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:28.125621 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:50:28.125941 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:28.125749 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-metrics-tls podName:fa1690c9-bef6-43d0-9aff-2359f4dfc39f nodeName:}" failed. No retries permitted until 2026-07-07 18:50:28.625729656 +0000 UTC m=+40.595365668 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-metrics-tls") pod "dns-default-b57q9" (UID: "fa1690c9-bef6-43d0-9aff-2359f4dfc39f") : secret "dns-default-metrics-tls" not found Jul 07 18:50:28.125996 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:28.125977 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-tmp-dir\") pod \"dns-default-b57q9\" (UID: \"fa1690c9-bef6-43d0-9aff-2359f4dfc39f\") " pod="openshift-dns/dns-default-b57q9" Jul 07 18:50:28.126220 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:28.126183 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-config-volume\") pod \"dns-default-b57q9\" (UID: \"fa1690c9-bef6-43d0-9aff-2359f4dfc39f\") " pod="openshift-dns/dns-default-b57q9" Jul 07 18:50:28.129881 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:28.129848 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf6f4b56-2567-469f-a0d4-bc586ab3c6bf-cert\") pod \"ingress-canary-bphf7\" (UID: \"cf6f4b56-2567-469f-a0d4-bc586ab3c6bf\") " pod="openshift-ingress-canary/ingress-canary-bphf7" Jul 07 18:50:28.135558 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:28.135535 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2wnh\" (UniqueName: \"kubernetes.io/projected/cf6f4b56-2567-469f-a0d4-bc586ab3c6bf-kube-api-access-z2wnh\") pod \"ingress-canary-bphf7\" (UID: \"cf6f4b56-2567-469f-a0d4-bc586ab3c6bf\") " pod="openshift-ingress-canary/ingress-canary-bphf7" Jul 07 18:50:28.136089 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:28.136072 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfkj9\" (UniqueName: \"kubernetes.io/projected/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-kube-api-access-sfkj9\") pod \"dns-default-b57q9\" (UID: \"fa1690c9-bef6-43d0-9aff-2359f4dfc39f\") " pod="openshift-dns/dns-default-b57q9" Jul 07 18:50:28.229340 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:28.229314 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bphf7" Jul 07 18:50:28.422869 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:28.422648 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bphf7"] Jul 07 18:50:28.431387 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:50:28.431361 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf6f4b56_2567_469f_a0d4_bc586ab3c6bf.slice/crio-46236dd3d1d58b2020b2119d03a6076fec9c2c9ce3933820237e124ef71f7304 WatchSource:0}: Error finding container 46236dd3d1d58b2020b2119d03a6076fec9c2c9ce3933820237e124ef71f7304: Status 404 returned error can't find the container with id 46236dd3d1d58b2020b2119d03a6076fec9c2c9ce3933820237e124ef71f7304 Jul 07 18:50:28.628046 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:28.628022 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-metrics-tls\") pod \"dns-default-b57q9\" (UID: \"fa1690c9-bef6-43d0-9aff-2359f4dfc39f\") " pod="openshift-dns/dns-default-b57q9" Jul 07 18:50:28.628211 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:28.628178 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:50:28.628291 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:28.628281 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-metrics-tls podName:fa1690c9-bef6-43d0-9aff-2359f4dfc39f nodeName:}" failed. No retries permitted until 2026-07-07 18:50:29.628265007 +0000 UTC m=+41.597901004 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-metrics-tls") pod "dns-default-b57q9" (UID: "fa1690c9-bef6-43d0-9aff-2359f4dfc39f") : secret "dns-default-metrics-tls" not found Jul 07 18:50:28.841330 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:28.841302 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bphf7" event={"ID":"cf6f4b56-2567-469f-a0d4-bc586ab3c6bf","Type":"ContainerStarted","Data":"46236dd3d1d58b2020b2119d03a6076fec9c2c9ce3933820237e124ef71f7304"} Jul 07 18:50:29.567982 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:29.567897 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:50:29.567982 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:29.567935 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:50:29.568247 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:29.568016 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:50:29.571853 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:29.571660 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 18:50:29.571853 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:29.571669 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x88g7\"" Jul 07 18:50:29.571853 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:29.571681 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7mdbh\"" Jul 07 18:50:29.571853 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:29.571698 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 18:50:29.571853 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:29.571699 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 18:50:29.571853 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:29.571803 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 18:50:29.634992 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:29.634962 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-metrics-tls\") pod \"dns-default-b57q9\" (UID: \"fa1690c9-bef6-43d0-9aff-2359f4dfc39f\") " pod="openshift-dns/dns-default-b57q9" Jul 07 18:50:29.635171 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:29.635073 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:50:29.635171 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:29.635136 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-metrics-tls podName:fa1690c9-bef6-43d0-9aff-2359f4dfc39f nodeName:}" failed. No retries permitted until 2026-07-07 18:50:31.635117821 +0000 UTC m=+43.604753818 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-metrics-tls") pod "dns-default-b57q9" (UID: "fa1690c9-bef6-43d0-9aff-2359f4dfc39f") : secret "dns-default-metrics-tls" not found Jul 07 18:50:30.312273 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.312237 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-cxwq7"] Jul 07 18:50:30.328425 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.328400 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-cxwq7"] Jul 07 18:50:30.328579 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.328525 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-cxwq7" Jul 07 18:50:30.331042 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.331022 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 07 18:50:30.331266 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.331251 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 07 18:50:30.331348 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.331304 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-zcwfd\"" Jul 07 18:50:30.438426 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.438396 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-trfsb"] Jul 07 18:50:30.439659 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.439625 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9250496d-2e44-489b-b3c0-b6c3121077a3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-cxwq7\" (UID: \"9250496d-2e44-489b-b3c0-b6c3121077a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cxwq7" Jul 07 18:50:30.439810 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.439712 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9250496d-2e44-489b-b3c0-b6c3121077a3-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-cxwq7\" (UID: \"9250496d-2e44-489b-b3c0-b6c3121077a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cxwq7" Jul 07 18:50:30.446471 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.446447 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d4mz6"] Jul 07 18:50:30.446627 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.446611 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" Jul 07 18:50:30.448669 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.448650 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 18:50:30.448795 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.448723 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-mrbcc\"" Jul 07 18:50:30.449023 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.448996 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 18:50:30.449307 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.449289 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 07 18:50:30.449405 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.449343 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 07 18:50:30.454684 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.454665 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d4mz6" Jul 07 18:50:30.459635 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.459616 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 07 18:50:30.459730 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.459654 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 07 18:50:30.459730 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.459665 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:50:30.459730 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.459677 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 07 18:50:30.459884 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.459834 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-2b56m\"" Jul 07 18:50:30.461735 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.461718 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 07 18:50:30.475036 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.475006 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d4mz6"] Jul 07 18:50:30.475849 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.475832 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-trfsb"] Jul 07 18:50:30.540212 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.540168 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9250496d-2e44-489b-b3c0-b6c3121077a3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-cxwq7\" (UID: \"9250496d-2e44-489b-b3c0-b6c3121077a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cxwq7" Jul 07 18:50:30.540329 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.540269 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9250496d-2e44-489b-b3c0-b6c3121077a3-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-cxwq7\" (UID: \"9250496d-2e44-489b-b3c0-b6c3121077a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cxwq7" Jul 07 18:50:30.540329 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.540316 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ccc5d0f-1bc6-4866-b99a-82252b234d0f-tmp\") pod \"insights-operator-5c847dcfc4-trfsb\" (UID: \"1ccc5d0f-1bc6-4866-b99a-82252b234d0f\") " pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" Jul 07 18:50:30.540430 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.540335 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7049298b-743d-4673-a84f-4e2465166aa3-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-d4mz6\" (UID: \"7049298b-743d-4673-a84f-4e2465166aa3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d4mz6" Jul 07 18:50:30.540430 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:30.540316 2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 18:50:30.540430 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.540378 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1ccc5d0f-1bc6-4866-b99a-82252b234d0f-snapshots\") pod \"insights-operator-5c847dcfc4-trfsb\" (UID: \"1ccc5d0f-1bc6-4866-b99a-82252b234d0f\") " pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" Jul 07 18:50:30.540430 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.540401 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ccc5d0f-1bc6-4866-b99a-82252b234d0f-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-trfsb\" (UID: \"1ccc5d0f-1bc6-4866-b99a-82252b234d0f\") " pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" Jul 07 18:50:30.540430 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.540420 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ccc5d0f-1bc6-4866-b99a-82252b234d0f-serving-cert\") pod \"insights-operator-5c847dcfc4-trfsb\" (UID: \"1ccc5d0f-1bc6-4866-b99a-82252b234d0f\") " pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" Jul 07 18:50:30.540670 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:30.540446 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9250496d-2e44-489b-b3c0-b6c3121077a3-networking-console-plugin-cert podName:9250496d-2e44-489b-b3c0-b6c3121077a3 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:31.040422235 +0000 UTC m=+43.010058244 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9250496d-2e44-489b-b3c0-b6c3121077a3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-cxwq7" (UID: "9250496d-2e44-489b-b3c0-b6c3121077a3") : secret "networking-console-plugin-cert" not found Jul 07 18:50:30.540670 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.540503 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ccc5d0f-1bc6-4866-b99a-82252b234d0f-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-trfsb\" (UID: \"1ccc5d0f-1bc6-4866-b99a-82252b234d0f\") " pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" Jul 07 18:50:30.540670 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.540547 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q42hn\" (UniqueName: \"kubernetes.io/projected/1ccc5d0f-1bc6-4866-b99a-82252b234d0f-kube-api-access-q42hn\") pod \"insights-operator-5c847dcfc4-trfsb\" (UID: \"1ccc5d0f-1bc6-4866-b99a-82252b234d0f\") " pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" Jul 07 18:50:30.540670 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.540589 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7049298b-743d-4673-a84f-4e2465166aa3-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-d4mz6\" (UID: \"7049298b-743d-4673-a84f-4e2465166aa3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d4mz6" Jul 07 18:50:30.540670 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.540635 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24b5v\" (UniqueName: \"kubernetes.io/projected/7049298b-743d-4673-a84f-4e2465166aa3-kube-api-access-24b5v\") pod \"kube-storage-version-migrator-operator-846bf668f7-d4mz6\" (UID: \"7049298b-743d-4673-a84f-4e2465166aa3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d4mz6" Jul 07 18:50:30.541057 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.541036 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9250496d-2e44-489b-b3c0-b6c3121077a3-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-cxwq7\" (UID: \"9250496d-2e44-489b-b3c0-b6c3121077a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cxwq7" Jul 07 18:50:30.542915 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.542896 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs"] Jul 07 18:50:30.559317 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.559190 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs" Jul 07 18:50:30.561536 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.561513 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs"] Jul 07 18:50:30.562616 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.562551 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 07 18:50:30.562800 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.562785 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-98vpd\"" Jul 07 18:50:30.563166 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.563026 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 07 18:50:30.563166 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.563095 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 18:50:30.563671 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.563652 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 18:50:30.630826 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.630777 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-569b766d4c-sdn6q"] Jul 07 18:50:30.641061 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.641040 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ccc5d0f-1bc6-4866-b99a-82252b234d0f-tmp\") pod \"insights-operator-5c847dcfc4-trfsb\" (UID: \"1ccc5d0f-1bc6-4866-b99a-82252b234d0f\") " pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" Jul 07 18:50:30.641190 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.641076 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7049298b-743d-4673-a84f-4e2465166aa3-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-d4mz6\" (UID: \"7049298b-743d-4673-a84f-4e2465166aa3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d4mz6" Jul 07 18:50:30.641190 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.641121 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1ccc5d0f-1bc6-4866-b99a-82252b234d0f-snapshots\") pod \"insights-operator-5c847dcfc4-trfsb\" (UID: \"1ccc5d0f-1bc6-4866-b99a-82252b234d0f\") " pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" Jul 07 18:50:30.641190 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.641145 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ccc5d0f-1bc6-4866-b99a-82252b234d0f-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-trfsb\" (UID: \"1ccc5d0f-1bc6-4866-b99a-82252b234d0f\") " pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" Jul 07 18:50:30.641190 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.641175 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ccc5d0f-1bc6-4866-b99a-82252b234d0f-serving-cert\") pod \"insights-operator-5c847dcfc4-trfsb\" (UID: \"1ccc5d0f-1bc6-4866-b99a-82252b234d0f\") " pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" Jul 07 18:50:30.641423 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.641214 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ccc5d0f-1bc6-4866-b99a-82252b234d0f-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-trfsb\" (UID: \"1ccc5d0f-1bc6-4866-b99a-82252b234d0f\") " pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" Jul 07 18:50:30.641423 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.641238 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q42hn\" (UniqueName: \"kubernetes.io/projected/1ccc5d0f-1bc6-4866-b99a-82252b234d0f-kube-api-access-q42hn\") pod \"insights-operator-5c847dcfc4-trfsb\" (UID: \"1ccc5d0f-1bc6-4866-b99a-82252b234d0f\") " pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" Jul 07 18:50:30.641423 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.641269 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-m8fgs\" (UID: \"64e169d3-cc4a-46a0-bfe6-1fb28c8deeae\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs" Jul 07 18:50:30.641423 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.641321 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7049298b-743d-4673-a84f-4e2465166aa3-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-d4mz6\" (UID: \"7049298b-743d-4673-a84f-4e2465166aa3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d4mz6" Jul 07 18:50:30.641423 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.641349 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24b5v\" (UniqueName: \"kubernetes.io/projected/7049298b-743d-4673-a84f-4e2465166aa3-kube-api-access-24b5v\") pod \"kube-storage-version-migrator-operator-846bf668f7-d4mz6\" (UID: \"7049298b-743d-4673-a84f-4e2465166aa3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d4mz6" Jul 07 18:50:30.641423 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.641386 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-m8fgs\" (UID: \"64e169d3-cc4a-46a0-bfe6-1fb28c8deeae\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs" Jul 07 18:50:30.641696 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.641501 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p5s5\" (UniqueName: \"kubernetes.io/projected/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-kube-api-access-5p5s5\") pod \"cluster-monitoring-operator-64786d9cbb-m8fgs\" (UID: \"64e169d3-cc4a-46a0-bfe6-1fb28c8deeae\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs" Jul 07 18:50:30.642446 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.641920 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1ccc5d0f-1bc6-4866-b99a-82252b234d0f-tmp\") pod \"insights-operator-5c847dcfc4-trfsb\" (UID: \"1ccc5d0f-1bc6-4866-b99a-82252b234d0f\") " pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" Jul 07 18:50:30.642446 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.642264 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1ccc5d0f-1bc6-4866-b99a-82252b234d0f-snapshots\") pod \"insights-operator-5c847dcfc4-trfsb\" (UID: \"1ccc5d0f-1bc6-4866-b99a-82252b234d0f\") " pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" Jul 07 18:50:30.642446 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.642386 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7049298b-743d-4673-a84f-4e2465166aa3-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-d4mz6\" (UID: \"7049298b-743d-4673-a84f-4e2465166aa3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d4mz6" Jul 07 18:50:30.642446 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.642405 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ccc5d0f-1bc6-4866-b99a-82252b234d0f-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-trfsb\" (UID: \"1ccc5d0f-1bc6-4866-b99a-82252b234d0f\") " pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" Jul 07 18:50:30.642643 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.642584 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ccc5d0f-1bc6-4866-b99a-82252b234d0f-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-trfsb\" (UID: \"1ccc5d0f-1bc6-4866-b99a-82252b234d0f\") " pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" Jul 07 18:50:30.645237 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.645215 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7049298b-743d-4673-a84f-4e2465166aa3-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-d4mz6\" (UID: \"7049298b-743d-4673-a84f-4e2465166aa3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d4mz6" Jul 07 18:50:30.645348 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.645215 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ccc5d0f-1bc6-4866-b99a-82252b234d0f-serving-cert\") pod \"insights-operator-5c847dcfc4-trfsb\" (UID: \"1ccc5d0f-1bc6-4866-b99a-82252b234d0f\") " pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" Jul 07 18:50:30.654159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.654135 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24b5v\" (UniqueName: \"kubernetes.io/projected/7049298b-743d-4673-a84f-4e2465166aa3-kube-api-access-24b5v\") pod \"kube-storage-version-migrator-operator-846bf668f7-d4mz6\" (UID: \"7049298b-743d-4673-a84f-4e2465166aa3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d4mz6" Jul 07 18:50:30.665889 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.665868 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q42hn\" (UniqueName: \"kubernetes.io/projected/1ccc5d0f-1bc6-4866-b99a-82252b234d0f-kube-api-access-q42hn\") pod \"insights-operator-5c847dcfc4-trfsb\" (UID: \"1ccc5d0f-1bc6-4866-b99a-82252b234d0f\") " pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" Jul 07 18:50:30.666277 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.666258 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-569b766d4c-sdn6q"] Jul 07 18:50:30.666384 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.666284 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.669691 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.669675 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 18:50:30.669865 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.669837 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4c4xx\"" Jul 07 18:50:30.670090 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.670071 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 18:50:30.670299 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.670285 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 18:50:30.676450 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.676430 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 18:50:30.742013 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.741986 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-m8fgs\" (UID: \"64e169d3-cc4a-46a0-bfe6-1fb28c8deeae\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs" Jul 07 18:50:30.742128 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.742037 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-m8fgs\" (UID: \"64e169d3-cc4a-46a0-bfe6-1fb28c8deeae\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs" Jul 07 18:50:30.742222 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.742163 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p5s5\" (UniqueName: \"kubernetes.io/projected/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-kube-api-access-5p5s5\") pod \"cluster-monitoring-operator-64786d9cbb-m8fgs\" (UID: \"64e169d3-cc4a-46a0-bfe6-1fb28c8deeae\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs" Jul 07 18:50:30.742285 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:30.742238 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:30.742344 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:30.742307 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-cluster-monitoring-operator-tls podName:64e169d3-cc4a-46a0-bfe6-1fb28c8deeae nodeName:}" failed. No retries permitted until 2026-07-07 18:50:31.242287968 +0000 UTC m=+43.211923981 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-m8fgs" (UID: "64e169d3-cc4a-46a0-bfe6-1fb28c8deeae") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:30.742751 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.742726 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-m8fgs\" (UID: \"64e169d3-cc4a-46a0-bfe6-1fb28c8deeae\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs" Jul 07 18:50:30.755957 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.755940 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p5s5\" (UniqueName: \"kubernetes.io/projected/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-kube-api-access-5p5s5\") pod \"cluster-monitoring-operator-64786d9cbb-m8fgs\" (UID: \"64e169d3-cc4a-46a0-bfe6-1fb28c8deeae\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs" Jul 07 18:50:30.761717 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.761680 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" Jul 07 18:50:30.768981 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.768962 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d4mz6" Jul 07 18:50:30.843419 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.843355 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2tmz\" (UniqueName: \"kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-kube-api-access-f2tmz\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.843419 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.843394 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-tls\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.843586 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.843446 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fe234b89-9fa1-4515-a421-e5f3e4393eae-ca-trust-extracted\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.843586 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.843542 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-certificates\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.843674 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.843633 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fe234b89-9fa1-4515-a421-e5f3e4393eae-image-registry-private-configuration\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.843727 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.843687 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-bound-sa-token\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.843727 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.843712 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fe234b89-9fa1-4515-a421-e5f3e4393eae-installation-pull-secrets\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.843857 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.843837 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fe234b89-9fa1-4515-a421-e5f3e4393eae-trusted-ca\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.944698 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.944664 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fe234b89-9fa1-4515-a421-e5f3e4393eae-trusted-ca\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.944863 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.944726 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2tmz\" (UniqueName: \"kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-kube-api-access-f2tmz\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.944863 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.944753 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-tls\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.944863 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.944799 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fe234b89-9fa1-4515-a421-e5f3e4393eae-ca-trust-extracted\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.945030 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:30.944907 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 18:50:30.945030 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.944918 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-certificates\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.945030 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.944950 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fe234b89-9fa1-4515-a421-e5f3e4393eae-image-registry-private-configuration\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.945030 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.944992 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-bound-sa-token\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.945030 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.945026 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fe234b89-9fa1-4515-a421-e5f3e4393eae-installation-pull-secrets\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.945306 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:30.944928 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-569b766d4c-sdn6q: secret "image-registry-tls" not found Jul 07 18:50:30.945306 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:30.945219 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-tls podName:fe234b89-9fa1-4515-a421-e5f3e4393eae nodeName:}" failed. No retries permitted until 2026-07-07 18:50:31.445173882 +0000 UTC m=+43.414809893 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-tls") pod "image-registry-569b766d4c-sdn6q" (UID: "fe234b89-9fa1-4515-a421-e5f3e4393eae") : secret "image-registry-tls" not found Jul 07 18:50:30.945306 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.945230 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fe234b89-9fa1-4515-a421-e5f3e4393eae-ca-trust-extracted\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.945578 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.945559 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-certificates\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.945651 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.945634 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fe234b89-9fa1-4515-a421-e5f3e4393eae-trusted-ca\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.947830 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.947810 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fe234b89-9fa1-4515-a421-e5f3e4393eae-installation-pull-secrets\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.947911 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.947873 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fe234b89-9fa1-4515-a421-e5f3e4393eae-image-registry-private-configuration\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.954952 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.954921 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-bound-sa-token\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:30.955101 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:30.955018 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2tmz\" (UniqueName: \"kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-kube-api-access-f2tmz\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:31.046191 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:31.046163 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9250496d-2e44-489b-b3c0-b6c3121077a3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-cxwq7\" (UID: \"9250496d-2e44-489b-b3c0-b6c3121077a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cxwq7" Jul 07 18:50:31.046335 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:31.046303 2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 18:50:31.046384 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:31.046372 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9250496d-2e44-489b-b3c0-b6c3121077a3-networking-console-plugin-cert podName:9250496d-2e44-489b-b3c0-b6c3121077a3 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:32.04635753 +0000 UTC m=+44.015993526 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9250496d-2e44-489b-b3c0-b6c3121077a3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-cxwq7" (UID: "9250496d-2e44-489b-b3c0-b6c3121077a3") : secret "networking-console-plugin-cert" not found Jul 07 18:50:31.213705 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:31.213670 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-trfsb"] Jul 07 18:50:31.214470 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:31.214450 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d4mz6"] Jul 07 18:50:31.216917 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:50:31.216890 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ccc5d0f_1bc6_4866_b99a_82252b234d0f.slice/crio-222653f3742f37429fcaa1aabff94353e544d62db66d8eb8ceb43d3420aa1c77 WatchSource:0}: Error finding container 222653f3742f37429fcaa1aabff94353e544d62db66d8eb8ceb43d3420aa1c77: Status 404 returned error can't find the container with id 222653f3742f37429fcaa1aabff94353e544d62db66d8eb8ceb43d3420aa1c77 Jul 07 18:50:31.217116 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:50:31.217097 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7049298b_743d_4673_a84f_4e2465166aa3.slice/crio-69735aff4906dda0ed60b8819a888ec1e7b4f6c65975e358ed0b2ef59c0ab429 WatchSource:0}: Error finding container 69735aff4906dda0ed60b8819a888ec1e7b4f6c65975e358ed0b2ef59c0ab429: Status 404 returned error can't find the container with id 69735aff4906dda0ed60b8819a888ec1e7b4f6c65975e358ed0b2ef59c0ab429 Jul 07 18:50:31.248442 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:31.248418 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-m8fgs\" (UID: \"64e169d3-cc4a-46a0-bfe6-1fb28c8deeae\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs" Jul 07 18:50:31.248573 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:31.248559 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:31.248630 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:31.248621 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-cluster-monitoring-operator-tls podName:64e169d3-cc4a-46a0-bfe6-1fb28c8deeae nodeName:}" failed. No retries permitted until 2026-07-07 18:50:32.248604366 +0000 UTC m=+44.218240376 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-m8fgs" (UID: "64e169d3-cc4a-46a0-bfe6-1fb28c8deeae") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:31.449818 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:31.449794 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-tls\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:31.450127 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:31.449942 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 18:50:31.450127 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:31.449960 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-569b766d4c-sdn6q: secret "image-registry-tls" not found Jul 07 18:50:31.450127 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:31.450024 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-tls podName:fe234b89-9fa1-4515-a421-e5f3e4393eae nodeName:}" failed. No retries permitted until 2026-07-07 18:50:32.450008438 +0000 UTC m=+44.419644453 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-tls") pod "image-registry-569b766d4c-sdn6q" (UID: "fe234b89-9fa1-4515-a421-e5f3e4393eae") : secret "image-registry-tls" not found Jul 07 18:50:31.652153 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:31.651958 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-metrics-tls\") pod \"dns-default-b57q9\" (UID: \"fa1690c9-bef6-43d0-9aff-2359f4dfc39f\") " pod="openshift-dns/dns-default-b57q9" Jul 07 18:50:31.652153 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:31.652132 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:50:31.652391 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:31.652233 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-metrics-tls podName:fa1690c9-bef6-43d0-9aff-2359f4dfc39f nodeName:}" failed. No retries permitted until 2026-07-07 18:50:35.652191205 +0000 UTC m=+47.621827212 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-metrics-tls") pod "dns-default-b57q9" (UID: "fa1690c9-bef6-43d0-9aff-2359f4dfc39f") : secret "dns-default-metrics-tls" not found Jul 07 18:50:31.850902 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:31.850753 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bphf7" event={"ID":"cf6f4b56-2567-469f-a0d4-bc586ab3c6bf","Type":"ContainerStarted","Data":"42f7203c5940e114bbe37a2861ca484ef05d3c84530e8d935a30a077b20e709d"} Jul 07 18:50:31.852977 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:31.852943 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" event={"ID":"1ccc5d0f-1bc6-4866-b99a-82252b234d0f","Type":"ContainerStarted","Data":"222653f3742f37429fcaa1aabff94353e544d62db66d8eb8ceb43d3420aa1c77"} Jul 07 18:50:31.854494 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:31.854463 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d4mz6" event={"ID":"7049298b-743d-4673-a84f-4e2465166aa3","Type":"ContainerStarted","Data":"69735aff4906dda0ed60b8819a888ec1e7b4f6c65975e358ed0b2ef59c0ab429"} Jul 07 18:50:31.873919 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:31.873837 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bphf7" podStartSLOduration=1.8422040530000001 podStartE2EDuration="4.873819918s" totalImagesPullingTime="3.031615865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:31.873404198 +0000 UTC m=+43.843040249" watchObservedRunningTime="2026-07-07 18:50:31.873819918 +0000 UTC m=+43.843455947" Jul 07 18:50:32.054491 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:32.054455 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9250496d-2e44-489b-b3c0-b6c3121077a3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-cxwq7\" (UID: \"9250496d-2e44-489b-b3c0-b6c3121077a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cxwq7" Jul 07 18:50:32.054670 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:32.054647 2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 18:50:32.054728 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:32.054712 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9250496d-2e44-489b-b3c0-b6c3121077a3-networking-console-plugin-cert podName:9250496d-2e44-489b-b3c0-b6c3121077a3 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:34.054694754 +0000 UTC m=+46.024330764 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9250496d-2e44-489b-b3c0-b6c3121077a3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-cxwq7" (UID: "9250496d-2e44-489b-b3c0-b6c3121077a3") : secret "networking-console-plugin-cert" not found Jul 07 18:50:32.256394 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:32.256356 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-m8fgs\" (UID: \"64e169d3-cc4a-46a0-bfe6-1fb28c8deeae\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs" Jul 07 18:50:32.256552 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:32.256511 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:32.256612 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:32.256599 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-cluster-monitoring-operator-tls podName:64e169d3-cc4a-46a0-bfe6-1fb28c8deeae nodeName:}" failed. No retries permitted until 2026-07-07 18:50:34.256578303 +0000 UTC m=+46.226214302 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-m8fgs" (UID: "64e169d3-cc4a-46a0-bfe6-1fb28c8deeae") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:32.459027 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:32.458996 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-tls\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:32.459561 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:32.459166 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 18:50:32.459561 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:32.459188 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-569b766d4c-sdn6q: secret "image-registry-tls" not found Jul 07 18:50:32.459561 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:32.459265 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-tls podName:fe234b89-9fa1-4515-a421-e5f3e4393eae nodeName:}" failed. No retries permitted until 2026-07-07 18:50:34.4592458 +0000 UTC m=+46.428881800 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-tls") pod "image-registry-569b766d4c-sdn6q" (UID: "fe234b89-9fa1-4515-a421-e5f3e4393eae") : secret "image-registry-tls" not found Jul 07 18:50:34.071495 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:34.071456 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9250496d-2e44-489b-b3c0-b6c3121077a3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-cxwq7\" (UID: \"9250496d-2e44-489b-b3c0-b6c3121077a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cxwq7" Jul 07 18:50:34.072076 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:34.071629 2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 18:50:34.072076 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:34.071710 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9250496d-2e44-489b-b3c0-b6c3121077a3-networking-console-plugin-cert podName:9250496d-2e44-489b-b3c0-b6c3121077a3 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:38.071688291 +0000 UTC m=+50.041324298 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9250496d-2e44-489b-b3c0-b6c3121077a3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-cxwq7" (UID: "9250496d-2e44-489b-b3c0-b6c3121077a3") : secret "networking-console-plugin-cert" not found Jul 07 18:50:34.273493 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:34.273458 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-m8fgs\" (UID: \"64e169d3-cc4a-46a0-bfe6-1fb28c8deeae\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs" Jul 07 18:50:34.273655 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:34.273573 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:34.273655 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:34.273625 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-cluster-monitoring-operator-tls podName:64e169d3-cc4a-46a0-bfe6-1fb28c8deeae nodeName:}" failed. No retries permitted until 2026-07-07 18:50:38.273611465 +0000 UTC m=+50.243247460 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-m8fgs" (UID: "64e169d3-cc4a-46a0-bfe6-1fb28c8deeae") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:34.475396 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:34.475370 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-tls\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:34.475555 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:34.475536 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 18:50:34.475606 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:34.475559 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-569b766d4c-sdn6q: secret "image-registry-tls" not found Jul 07 18:50:34.475647 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:34.475617 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-tls podName:fe234b89-9fa1-4515-a421-e5f3e4393eae nodeName:}" failed. No retries permitted until 2026-07-07 18:50:38.475597894 +0000 UTC m=+50.445233906 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-tls") pod "image-registry-569b766d4c-sdn6q" (UID: "fe234b89-9fa1-4515-a421-e5f3e4393eae") : secret "image-registry-tls" not found Jul 07 18:50:34.863842 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:34.863800 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d4mz6" event={"ID":"7049298b-743d-4673-a84f-4e2465166aa3","Type":"ContainerStarted","Data":"f06c1d31a57cf0a273182e0f807670bf855ca1331892a2759f5600a4428cc71d"} Jul 07 18:50:34.865335 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:34.865300 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" event={"ID":"1ccc5d0f-1bc6-4866-b99a-82252b234d0f","Type":"ContainerStarted","Data":"f105ef8f12c235f57694629feadbe50efae55990a60b58b08d490d532741f79a"} Jul 07 18:50:34.883771 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:34.883727 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d4mz6" podStartSLOduration=1.668242796 podStartE2EDuration="4.883711773s" totalImagesPullingTime="3.215468977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:34.883400241 +0000 UTC m=+46.853036263" watchObservedRunningTime="2026-07-07 18:50:34.883711773 +0000 UTC m=+46.853347789" Jul 07 18:50:34.910812 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:34.910764 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" podStartSLOduration=1.688957588 podStartE2EDuration="4.910745327s" totalImagesPullingTime="3.221787739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:34.910391238 +0000 UTC m=+46.880027280" watchObservedRunningTime="2026-07-07 18:50:34.910745327 +0000 UTC m=+46.880381349" Jul 07 18:50:35.688266 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:35.688231 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-metrics-tls\") pod \"dns-default-b57q9\" (UID: \"fa1690c9-bef6-43d0-9aff-2359f4dfc39f\") " pod="openshift-dns/dns-default-b57q9" Jul 07 18:50:35.688819 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:35.688385 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:50:35.688819 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:35.688459 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-metrics-tls podName:fa1690c9-bef6-43d0-9aff-2359f4dfc39f nodeName:}" failed. No retries permitted until 2026-07-07 18:50:43.68844335 +0000 UTC m=+55.658079348 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-metrics-tls") pod "dns-default-b57q9" (UID: "fa1690c9-bef6-43d0-9aff-2359f4dfc39f") : secret "dns-default-metrics-tls" not found Jul 07 18:50:35.706498 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:35.706468 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lksfn"] Jul 07 18:50:35.727944 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:35.727914 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lksfn"] Jul 07 18:50:35.728114 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:35.728040 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:35.730755 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:35.730706 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 18:50:35.730755 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:35.730745 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 18:50:35.730921 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:35.730797 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ckhk8\"" Jul 07 18:50:35.890725 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:35.890698 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5a4777b9-2b1e-4987-9073-426919e6919c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:35.890903 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:35.890748 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw852\" (UniqueName: \"kubernetes.io/projected/5a4777b9-2b1e-4987-9073-426919e6919c-kube-api-access-zw852\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:35.890903 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:35.890840 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5a4777b9-2b1e-4987-9073-426919e6919c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:35.890903 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:35.890880 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5a4777b9-2b1e-4987-9073-426919e6919c-crio-socket\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:35.891011 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:35.890906 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a4777b9-2b1e-4987-9073-426919e6919c-data-volume\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:35.991766 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:35.991731 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5a4777b9-2b1e-4987-9073-426919e6919c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:35.991961 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:35.991855 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:50:35.991961 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:35.991936 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a4777b9-2b1e-4987-9073-426919e6919c-insights-runtime-extractor-tls podName:5a4777b9-2b1e-4987-9073-426919e6919c nodeName:}" failed. No retries permitted until 2026-07-07 18:50:36.491915366 +0000 UTC m=+48.461551373 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5a4777b9-2b1e-4987-9073-426919e6919c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lksfn" (UID: "5a4777b9-2b1e-4987-9073-426919e6919c") : secret "insights-runtime-extractor-tls" not found Jul 07 18:50:35.992090 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:35.991971 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zw852\" (UniqueName: \"kubernetes.io/projected/5a4777b9-2b1e-4987-9073-426919e6919c-kube-api-access-zw852\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:35.992090 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:35.992077 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5a4777b9-2b1e-4987-9073-426919e6919c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:35.992188 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:35.992153 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 18:50:35.992188 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:35.992173 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5a4777b9-2b1e-4987-9073-426919e6919c-crio-socket\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:35.992335 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:35.992245 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5a4777b9-2b1e-4987-9073-426919e6919c-kube-rbac-proxy-cm podName:5a4777b9-2b1e-4987-9073-426919e6919c nodeName:}" failed. No retries permitted until 2026-07-07 18:50:36.492228632 +0000 UTC m=+48.461864641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5a4777b9-2b1e-4987-9073-426919e6919c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lksfn" (UID: "5a4777b9-2b1e-4987-9073-426919e6919c") : configmap "kube-rbac-proxy" not found Jul 07 18:50:35.992335 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:35.992283 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a4777b9-2b1e-4987-9073-426919e6919c-data-volume\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:35.992426 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:35.992393 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5a4777b9-2b1e-4987-9073-426919e6919c-crio-socket\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:35.992574 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:35.992558 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a4777b9-2b1e-4987-9073-426919e6919c-data-volume\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:36.002446 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:36.002311 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw852\" (UniqueName: \"kubernetes.io/projected/5a4777b9-2b1e-4987-9073-426919e6919c-kube-api-access-zw852\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:36.497363 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:36.497330 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5a4777b9-2b1e-4987-9073-426919e6919c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:36.497531 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:36.497401 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5a4777b9-2b1e-4987-9073-426919e6919c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:36.497531 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:36.497490 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:50:36.497619 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:36.497554 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a4777b9-2b1e-4987-9073-426919e6919c-insights-runtime-extractor-tls podName:5a4777b9-2b1e-4987-9073-426919e6919c nodeName:}" failed. No retries permitted until 2026-07-07 18:50:37.497539596 +0000 UTC m=+49.467175594 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5a4777b9-2b1e-4987-9073-426919e6919c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lksfn" (UID: "5a4777b9-2b1e-4987-9073-426919e6919c") : secret "insights-runtime-extractor-tls" not found Jul 07 18:50:36.497619 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:36.497496 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 18:50:36.497693 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:36.497624 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5a4777b9-2b1e-4987-9073-426919e6919c-kube-rbac-proxy-cm podName:5a4777b9-2b1e-4987-9073-426919e6919c nodeName:}" failed. No retries permitted until 2026-07-07 18:50:37.497610225 +0000 UTC m=+49.467246228 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5a4777b9-2b1e-4987-9073-426919e6919c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lksfn" (UID: "5a4777b9-2b1e-4987-9073-426919e6919c") : configmap "kube-rbac-proxy" not found Jul 07 18:50:37.504108 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:37.504069 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5a4777b9-2b1e-4987-9073-426919e6919c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:37.504593 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:37.504211 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5a4777b9-2b1e-4987-9073-426919e6919c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:37.504593 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:37.504229 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:50:37.504593 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:37.504295 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a4777b9-2b1e-4987-9073-426919e6919c-insights-runtime-extractor-tls podName:5a4777b9-2b1e-4987-9073-426919e6919c nodeName:}" failed. No retries permitted until 2026-07-07 18:50:39.50427995 +0000 UTC m=+51.473915960 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5a4777b9-2b1e-4987-9073-426919e6919c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lksfn" (UID: "5a4777b9-2b1e-4987-9073-426919e6919c") : secret "insights-runtime-extractor-tls" not found Jul 07 18:50:37.504593 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:37.504305 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 18:50:37.504593 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:37.504359 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5a4777b9-2b1e-4987-9073-426919e6919c-kube-rbac-proxy-cm podName:5a4777b9-2b1e-4987-9073-426919e6919c nodeName:}" failed. No retries permitted until 2026-07-07 18:50:39.504345956 +0000 UTC m=+51.473981954 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5a4777b9-2b1e-4987-9073-426919e6919c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lksfn" (UID: "5a4777b9-2b1e-4987-9073-426919e6919c") : configmap "kube-rbac-proxy" not found Jul 07 18:50:37.762154 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:37.762084 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8bhnt_7dcc43c9-3758-47a3-aa8d-278166547dad/dns-node-resolver/0.log" Jul 07 18:50:38.110622 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:38.110540 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9250496d-2e44-489b-b3c0-b6c3121077a3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-cxwq7\" (UID: \"9250496d-2e44-489b-b3c0-b6c3121077a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cxwq7" Jul 07 18:50:38.110776 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:38.110692 2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 18:50:38.110776 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:38.110756 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9250496d-2e44-489b-b3c0-b6c3121077a3-networking-console-plugin-cert podName:9250496d-2e44-489b-b3c0-b6c3121077a3 nodeName:}" failed. No retries permitted until 2026-07-07 18:50:46.11074188 +0000 UTC m=+58.080377875 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9250496d-2e44-489b-b3c0-b6c3121077a3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-cxwq7" (UID: "9250496d-2e44-489b-b3c0-b6c3121077a3") : secret "networking-console-plugin-cert" not found Jul 07 18:50:38.312615 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:38.312578 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-m8fgs\" (UID: \"64e169d3-cc4a-46a0-bfe6-1fb28c8deeae\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs" Jul 07 18:50:38.312773 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:38.312729 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:38.312810 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:38.312791 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-cluster-monitoring-operator-tls podName:64e169d3-cc4a-46a0-bfe6-1fb28c8deeae nodeName:}" failed. No retries permitted until 2026-07-07 18:50:46.312775751 +0000 UTC m=+58.282411751 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-m8fgs" (UID: "64e169d3-cc4a-46a0-bfe6-1fb28c8deeae") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:38.513977 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:38.513937 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-tls\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:38.514372 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:38.514072 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 18:50:38.514372 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:38.514089 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-569b766d4c-sdn6q: secret "image-registry-tls" not found Jul 07 18:50:38.514372 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:38.514136 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-tls podName:fe234b89-9fa1-4515-a421-e5f3e4393eae nodeName:}" failed. No retries permitted until 2026-07-07 18:50:46.514122279 +0000 UTC m=+58.483758274 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-tls") pod "image-registry-569b766d4c-sdn6q" (UID: "fe234b89-9fa1-4515-a421-e5f3e4393eae") : secret "image-registry-tls" not found Jul 07 18:50:38.963509 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:38.963487 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qkhdg_8821d59a-1e1c-43d8-80d8-3513deb7478b/node-ca/0.log" Jul 07 18:50:39.361384 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:39.361299 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bphf7_cf6f4b56-2567-469f-a0d4-bc586ab3c6bf/serve-healthcheck-canary/0.log" Jul 07 18:50:39.522773 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:39.522729 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5a4777b9-2b1e-4987-9073-426919e6919c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:39.523237 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:39.522885 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:50:39.523237 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:39.522915 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5a4777b9-2b1e-4987-9073-426919e6919c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:39.523237 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:39.522952 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a4777b9-2b1e-4987-9073-426919e6919c-insights-runtime-extractor-tls podName:5a4777b9-2b1e-4987-9073-426919e6919c nodeName:}" failed. No retries permitted until 2026-07-07 18:50:43.522935817 +0000 UTC m=+55.492571815 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5a4777b9-2b1e-4987-9073-426919e6919c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lksfn" (UID: "5a4777b9-2b1e-4987-9073-426919e6919c") : secret "insights-runtime-extractor-tls" not found Jul 07 18:50:39.523237 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:39.522977 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 18:50:39.523237 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:39.523030 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5a4777b9-2b1e-4987-9073-426919e6919c-kube-rbac-proxy-cm podName:5a4777b9-2b1e-4987-9073-426919e6919c nodeName:}" failed. No retries permitted until 2026-07-07 18:50:43.52301709 +0000 UTC m=+55.492653102 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5a4777b9-2b1e-4987-9073-426919e6919c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lksfn" (UID: "5a4777b9-2b1e-4987-9073-426919e6919c") : configmap "kube-rbac-proxy" not found Jul 07 18:50:43.558723 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:43.558690 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5a4777b9-2b1e-4987-9073-426919e6919c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:43.559177 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:43.558813 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 18:50:43.559177 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:43.558812 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5a4777b9-2b1e-4987-9073-426919e6919c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:43.559177 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:43.558877 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5a4777b9-2b1e-4987-9073-426919e6919c-kube-rbac-proxy-cm podName:5a4777b9-2b1e-4987-9073-426919e6919c nodeName:}" failed. No retries permitted until 2026-07-07 18:50:51.558858645 +0000 UTC m=+63.528494657 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5a4777b9-2b1e-4987-9073-426919e6919c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lksfn" (UID: "5a4777b9-2b1e-4987-9073-426919e6919c") : configmap "kube-rbac-proxy" not found Jul 07 18:50:43.559177 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:43.558891 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 18:50:43.559177 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:43.558935 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a4777b9-2b1e-4987-9073-426919e6919c-insights-runtime-extractor-tls podName:5a4777b9-2b1e-4987-9073-426919e6919c nodeName:}" failed. No retries permitted until 2026-07-07 18:50:51.558923989 +0000 UTC m=+63.528559984 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5a4777b9-2b1e-4987-9073-426919e6919c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lksfn" (UID: "5a4777b9-2b1e-4987-9073-426919e6919c") : secret "insights-runtime-extractor-tls" not found Jul 07 18:50:43.761054 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:43.761016 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-metrics-tls\") pod \"dns-default-b57q9\" (UID: \"fa1690c9-bef6-43d0-9aff-2359f4dfc39f\") " pod="openshift-dns/dns-default-b57q9" Jul 07 18:50:43.761245 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:43.761165 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 18:50:43.761245 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:43.761244 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-metrics-tls podName:fa1690c9-bef6-43d0-9aff-2359f4dfc39f nodeName:}" failed. No retries permitted until 2026-07-07 18:50:59.761228692 +0000 UTC m=+71.730864687 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-metrics-tls") pod "dns-default-b57q9" (UID: "fa1690c9-bef6-43d0-9aff-2359f4dfc39f") : secret "dns-default-metrics-tls" not found Jul 07 18:50:46.178259 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:46.178216 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9250496d-2e44-489b-b3c0-b6c3121077a3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-cxwq7\" (UID: \"9250496d-2e44-489b-b3c0-b6c3121077a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cxwq7" Jul 07 18:50:46.178650 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:46.178365 2564 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 07 18:50:46.178650 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:46.178429 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9250496d-2e44-489b-b3c0-b6c3121077a3-networking-console-plugin-cert podName:9250496d-2e44-489b-b3c0-b6c3121077a3 nodeName:}" failed. No retries permitted until 2026-07-07 18:51:02.17841548 +0000 UTC m=+74.148051478 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9250496d-2e44-489b-b3c0-b6c3121077a3-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-cxwq7" (UID: "9250496d-2e44-489b-b3c0-b6c3121077a3") : secret "networking-console-plugin-cert" not found Jul 07 18:50:46.379324 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:46.379291 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-m8fgs\" (UID: \"64e169d3-cc4a-46a0-bfe6-1fb28c8deeae\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs" Jul 07 18:50:46.379529 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:46.379402 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:46.379529 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:50:46.379454 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-cluster-monitoring-operator-tls podName:64e169d3-cc4a-46a0-bfe6-1fb28c8deeae nodeName:}" failed. No retries permitted until 2026-07-07 18:51:02.379440987 +0000 UTC m=+74.349076981 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-m8fgs" (UID: "64e169d3-cc4a-46a0-bfe6-1fb28c8deeae") : secret "cluster-monitoring-operator-tls" not found Jul 07 18:50:46.581536 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:46.581508 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-tls\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:46.584497 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:46.584468 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-tls\") pod \"image-registry-569b766d4c-sdn6q\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:46.876738 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:46.876665 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:47.043589 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:47.043559 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-569b766d4c-sdn6q"] Jul 07 18:50:47.046612 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:50:47.046576 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe234b89_9fa1_4515_a421_e5f3e4393eae.slice/crio-45e6251028253c5134c235e247c12aa96fdb61dd5f3b595318c2415e132b5165 WatchSource:0}: Error finding container 45e6251028253c5134c235e247c12aa96fdb61dd5f3b595318c2415e132b5165: Status 404 returned error can't find the container with id 45e6251028253c5134c235e247c12aa96fdb61dd5f3b595318c2415e132b5165 Jul 07 18:50:47.897776 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:47.897735 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" event={"ID":"fe234b89-9fa1-4515-a421-e5f3e4393eae","Type":"ContainerStarted","Data":"2aa85128bf8496d0ebcfb4ee773bd6acc4fc8ca31be48e0a4f4201b53a954516"} Jul 07 18:50:47.898238 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:47.897787 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" event={"ID":"fe234b89-9fa1-4515-a421-e5f3e4393eae","Type":"ContainerStarted","Data":"45e6251028253c5134c235e247c12aa96fdb61dd5f3b595318c2415e132b5165"} Jul 07 18:50:47.898238 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:47.897868 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:50:47.919262 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:47.919219 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" podStartSLOduration=17.919184349 podStartE2EDuration="17.919184349s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:47.917838471 +0000 UTC m=+59.887474505" watchObservedRunningTime="2026-07-07 18:50:47.919184349 +0000 UTC m=+59.888820366" Jul 07 18:50:51.620076 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:51.620043 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5a4777b9-2b1e-4987-9073-426919e6919c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:51.620580 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:51.620238 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5a4777b9-2b1e-4987-9073-426919e6919c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:51.621283 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:51.621260 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5a4777b9-2b1e-4987-9073-426919e6919c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:51.622375 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:51.622357 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5a4777b9-2b1e-4987-9073-426919e6919c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lksfn\" (UID: \"5a4777b9-2b1e-4987-9073-426919e6919c\") " pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:51.639497 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:51.639478 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ckhk8\"" Jul 07 18:50:51.647390 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:51.647377 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lksfn" Jul 07 18:50:51.768409 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:51.768387 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lksfn"] Jul 07 18:50:51.771104 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:50:51.771080 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a4777b9_2b1e_4987_9073_426919e6919c.slice/crio-f167819f1b86e2f92c4513faec4e7dae065d6b91dc9bea9c0270571747d1b229 WatchSource:0}: Error finding container f167819f1b86e2f92c4513faec4e7dae065d6b91dc9bea9c0270571747d1b229: Status 404 returned error can't find the container with id f167819f1b86e2f92c4513faec4e7dae065d6b91dc9bea9c0270571747d1b229 Jul 07 18:50:51.826540 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:51.826516 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vkzgb" Jul 07 18:50:51.908322 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:51.908258 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lksfn" event={"ID":"5a4777b9-2b1e-4987-9073-426919e6919c","Type":"ContainerStarted","Data":"f167819f1b86e2f92c4513faec4e7dae065d6b91dc9bea9c0270571747d1b229"} Jul 07 18:50:54.241914 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:54.241873 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret\") pod \"global-pull-secret-syncer-ffm5c\" (UID: \"570bed30-5e16-475c-9cc4-4e5af148c2e5\") " pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:50:54.242383 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:54.242055 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs\") pod \"network-metrics-daemon-ccjd5\" (UID: \"536dbfa9-2261-4932-a922-e11a2a764168\") " pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:50:54.244555 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:54.244532 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 18:50:54.244555 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:54.244549 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 18:50:54.254868 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:54.254848 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/570bed30-5e16-475c-9cc4-4e5af148c2e5-original-pull-secret\") pod \"global-pull-secret-syncer-ffm5c\" (UID: \"570bed30-5e16-475c-9cc4-4e5af148c2e5\") " pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:50:54.254993 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:54.254964 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/536dbfa9-2261-4932-a922-e11a2a764168-metrics-certs\") pod \"network-metrics-daemon-ccjd5\" (UID: \"536dbfa9-2261-4932-a922-e11a2a764168\") " pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:50:54.343243 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:54.343190 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52v7l\" (UniqueName: \"kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l\") pod \"network-check-target-mkg6r\" (UID: \"88340235-0f7e-4e90-bf52-86b3f04aa9ff\") " pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:50:54.345881 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:54.345858 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 18:50:54.356001 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:54.355970 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 18:50:54.367439 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:54.367418 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52v7l\" (UniqueName: \"kubernetes.io/projected/88340235-0f7e-4e90-bf52-86b3f04aa9ff-kube-api-access-52v7l\") pod \"network-check-target-mkg6r\" (UID: \"88340235-0f7e-4e90-bf52-86b3f04aa9ff\") " pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:50:54.483616 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:54.483590 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7mdbh\"" Jul 07 18:50:54.489617 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:54.489594 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x88g7\"" Jul 07 18:50:54.491346 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:54.491331 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:50:54.494466 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:54.494414 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffm5c" Jul 07 18:50:54.497764 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:54.497744 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ccjd5" Jul 07 18:50:55.679423 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:55.679346 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ccjd5"] Jul 07 18:50:55.684864 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:50:55.684838 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod536dbfa9_2261_4932_a922_e11a2a764168.slice/crio-04dc971d6495c396a5b5eadead9c65890612739767227e26c156623a41e9e866 WatchSource:0}: Error finding container 04dc971d6495c396a5b5eadead9c65890612739767227e26c156623a41e9e866: Status 404 returned error can't find the container with id 04dc971d6495c396a5b5eadead9c65890612739767227e26c156623a41e9e866 Jul 07 18:50:55.905242 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:55.905160 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ffm5c"] Jul 07 18:50:55.910586 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:55.910562 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mkg6r"] Jul 07 18:50:55.915915 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:50:55.915882 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod570bed30_5e16_475c_9cc4_4e5af148c2e5.slice/crio-0685a35573feec7885d308428b5a2406d5d581984fd5a830952739e404b0499c WatchSource:0}: Error finding container 0685a35573feec7885d308428b5a2406d5d581984fd5a830952739e404b0499c: Status 404 returned error can't find the container with id 0685a35573feec7885d308428b5a2406d5d581984fd5a830952739e404b0499c Jul 07 18:50:55.916561 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:50:55.916535 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88340235_0f7e_4e90_bf52_86b3f04aa9ff.slice/crio-93a1f8f0a9c90a09e399122284de34f0be5f88986bdadfa1ebeb99649fb33497 WatchSource:0}: Error finding container 93a1f8f0a9c90a09e399122284de34f0be5f88986bdadfa1ebeb99649fb33497: Status 404 returned error can't find the container with id 93a1f8f0a9c90a09e399122284de34f0be5f88986bdadfa1ebeb99649fb33497 Jul 07 18:50:55.919796 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:55.919770 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ccjd5" event={"ID":"536dbfa9-2261-4932-a922-e11a2a764168","Type":"ContainerStarted","Data":"04dc971d6495c396a5b5eadead9c65890612739767227e26c156623a41e9e866"} Jul 07 18:50:55.921051 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:55.921031 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ffm5c" event={"ID":"570bed30-5e16-475c-9cc4-4e5af148c2e5","Type":"ContainerStarted","Data":"0685a35573feec7885d308428b5a2406d5d581984fd5a830952739e404b0499c"} Jul 07 18:50:56.933415 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:56.933376 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mkg6r" event={"ID":"88340235-0f7e-4e90-bf52-86b3f04aa9ff","Type":"ContainerStarted","Data":"93a1f8f0a9c90a09e399122284de34f0be5f88986bdadfa1ebeb99649fb33497"} Jul 07 18:50:56.936116 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:56.936083 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lksfn" event={"ID":"5a4777b9-2b1e-4987-9073-426919e6919c","Type":"ContainerStarted","Data":"61517e2254d16e61ff692287df0e8aff0f3a2b33a2fec05a9d8a1f016c11ad40"} Jul 07 18:50:56.936265 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:56.936141 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lksfn" event={"ID":"5a4777b9-2b1e-4987-9073-426919e6919c","Type":"ContainerStarted","Data":"8ac6c235353270c54bf344d9992332e703e8ad1112bc9ed57e4a99deb08bf84d"} Jul 07 18:50:57.943096 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:57.943003 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ccjd5" event={"ID":"536dbfa9-2261-4932-a922-e11a2a764168","Type":"ContainerStarted","Data":"71f4c69487b895143f6e8ac0e8d677e4a5c0c224c76b2be1196545f401550d70"} Jul 07 18:50:57.943096 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:57.943066 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ccjd5" event={"ID":"536dbfa9-2261-4932-a922-e11a2a764168","Type":"ContainerStarted","Data":"a5505db75810eb3f44e65839213ca280cb2db264315d525b1d6130fc4f55fe9f"} Jul 07 18:50:57.966940 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:57.966603 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ccjd5" podStartSLOduration=68.599924837 podStartE2EDuration="1m9.966581469s" totalImagesPullingTime="1.366656632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:49:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:50:57.9651233 +0000 UTC m=+69.934759318" watchObservedRunningTime="2026-07-07 18:50:57.966581469 +0000 UTC m=+69.936217486" Jul 07 18:50:58.499282 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:58.499250 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5667689847-7pjmn"] Jul 07 18:50:58.502330 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:58.502311 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5667689847-7pjmn" Jul 07 18:50:58.505516 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:58.505492 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 07 18:50:58.506396 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:58.506375 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 07 18:50:58.506499 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:58.506475 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 07 18:50:58.509319 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:58.509285 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 07 18:50:58.510319 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:58.510161 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-569b766d4c-sdn6q"] Jul 07 18:50:58.528823 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:58.528798 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5667689847-7pjmn"] Jul 07 18:50:58.579215 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:58.579166 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9873de59-358e-420f-ab91-dcd142bdc45b-klusterlet-config\") pod \"klusterlet-addon-workmgr-5667689847-7pjmn\" (UID: \"9873de59-358e-420f-ab91-dcd142bdc45b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5667689847-7pjmn" Jul 07 18:50:58.579372 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:58.579339 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgdnh\" (UniqueName: \"kubernetes.io/projected/9873de59-358e-420f-ab91-dcd142bdc45b-kube-api-access-tgdnh\") pod \"klusterlet-addon-workmgr-5667689847-7pjmn\" (UID: \"9873de59-358e-420f-ab91-dcd142bdc45b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5667689847-7pjmn" Jul 07 18:50:58.579422 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:58.579379 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9873de59-358e-420f-ab91-dcd142bdc45b-tmp\") pod \"klusterlet-addon-workmgr-5667689847-7pjmn\" (UID: \"9873de59-358e-420f-ab91-dcd142bdc45b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5667689847-7pjmn" Jul 07 18:50:58.680814 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:58.680784 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgdnh\" (UniqueName: \"kubernetes.io/projected/9873de59-358e-420f-ab91-dcd142bdc45b-kube-api-access-tgdnh\") pod \"klusterlet-addon-workmgr-5667689847-7pjmn\" (UID: \"9873de59-358e-420f-ab91-dcd142bdc45b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5667689847-7pjmn" Jul 07 18:50:58.680989 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:58.680819 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9873de59-358e-420f-ab91-dcd142bdc45b-tmp\") pod \"klusterlet-addon-workmgr-5667689847-7pjmn\" (UID: \"9873de59-358e-420f-ab91-dcd142bdc45b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5667689847-7pjmn" Jul 07 18:50:58.680989 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:58.680851 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9873de59-358e-420f-ab91-dcd142bdc45b-klusterlet-config\") pod \"klusterlet-addon-workmgr-5667689847-7pjmn\" (UID: \"9873de59-358e-420f-ab91-dcd142bdc45b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5667689847-7pjmn" Jul 07 18:50:58.681262 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:58.681239 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9873de59-358e-420f-ab91-dcd142bdc45b-tmp\") pod \"klusterlet-addon-workmgr-5667689847-7pjmn\" (UID: \"9873de59-358e-420f-ab91-dcd142bdc45b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5667689847-7pjmn" Jul 07 18:50:58.683742 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:58.683716 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/9873de59-358e-420f-ab91-dcd142bdc45b-klusterlet-config\") pod \"klusterlet-addon-workmgr-5667689847-7pjmn\" (UID: \"9873de59-358e-420f-ab91-dcd142bdc45b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5667689847-7pjmn" Jul 07 18:50:58.699090 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:58.699038 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgdnh\" (UniqueName: \"kubernetes.io/projected/9873de59-358e-420f-ab91-dcd142bdc45b-kube-api-access-tgdnh\") pod \"klusterlet-addon-workmgr-5667689847-7pjmn\" (UID: \"9873de59-358e-420f-ab91-dcd142bdc45b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5667689847-7pjmn" Jul 07 18:50:58.814706 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:58.814627 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5667689847-7pjmn" Jul 07 18:50:59.791211 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:59.791151 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-metrics-tls\") pod \"dns-default-b57q9\" (UID: \"fa1690c9-bef6-43d0-9aff-2359f4dfc39f\") " pod="openshift-dns/dns-default-b57q9" Jul 07 18:50:59.793683 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:50:59.793652 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fa1690c9-bef6-43d0-9aff-2359f4dfc39f-metrics-tls\") pod \"dns-default-b57q9\" (UID: \"fa1690c9-bef6-43d0-9aff-2359f4dfc39f\") " pod="openshift-dns/dns-default-b57q9" Jul 07 18:51:00.018179 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:00.018148 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-krxdw\"" Jul 07 18:51:00.026951 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:00.026931 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b57q9" Jul 07 18:51:01.972646 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:01.972595 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:51:01.994426 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:01.994374 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mkg6r" podStartSLOduration=68.076597572 podStartE2EDuration="1m13.994351143s" totalImagesPullingTime="5.917753571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:49:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:01.992130576 +0000 UTC m=+73.961766595" watchObservedRunningTime="2026-07-07 18:51:01.994351143 +0000 UTC m=+73.963987164" Jul 07 18:51:02.015227 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:02.015173 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b57q9"] Jul 07 18:51:02.017871 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:51:02.017844 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa1690c9_bef6_43d0_9aff_2359f4dfc39f.slice/crio-a2ac6096bdac2029ded0baed370940ee14a2f74c0f2f7e1a48c08dfc84099a7f WatchSource:0}: Error finding container a2ac6096bdac2029ded0baed370940ee14a2f74c0f2f7e1a48c08dfc84099a7f: Status 404 returned error can't find the container with id a2ac6096bdac2029ded0baed370940ee14a2f74c0f2f7e1a48c08dfc84099a7f Jul 07 18:51:02.029847 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:02.029818 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5667689847-7pjmn"] Jul 07 18:51:02.033502 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:51:02.033465 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9873de59_358e_420f_ab91_dcd142bdc45b.slice/crio-f66947ab44aba047187c9bb6d4581e123f750d76a08377c31c2c4f5f55b376fc WatchSource:0}: Error finding container f66947ab44aba047187c9bb6d4581e123f750d76a08377c31c2c4f5f55b376fc: Status 404 returned error can't find the container with id f66947ab44aba047187c9bb6d4581e123f750d76a08377c31c2c4f5f55b376fc Jul 07 18:51:02.211854 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:02.211814 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9250496d-2e44-489b-b3c0-b6c3121077a3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-cxwq7\" (UID: \"9250496d-2e44-489b-b3c0-b6c3121077a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cxwq7" Jul 07 18:51:02.214214 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:02.214172 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9250496d-2e44-489b-b3c0-b6c3121077a3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-cxwq7\" (UID: \"9250496d-2e44-489b-b3c0-b6c3121077a3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-cxwq7" Jul 07 18:51:02.413212 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:02.413171 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-m8fgs\" (UID: \"64e169d3-cc4a-46a0-bfe6-1fb28c8deeae\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs" Jul 07 18:51:02.415535 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:02.415514 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/64e169d3-cc4a-46a0-bfe6-1fb28c8deeae-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-m8fgs\" (UID: \"64e169d3-cc4a-46a0-bfe6-1fb28c8deeae\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs" Jul 07 18:51:02.440606 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:02.440574 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-zcwfd\"" Jul 07 18:51:02.448687 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:02.448668 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-cxwq7" Jul 07 18:51:02.590013 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:02.589987 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-cxwq7"] Jul 07 18:51:02.592506 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:51:02.592475 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9250496d_2e44_489b_b3c0_b6c3121077a3.slice/crio-a9c8e90858bf2e467e1ca745717616f43e9851925322c2c6293196a0d32961cc WatchSource:0}: Error finding container a9c8e90858bf2e467e1ca745717616f43e9851925322c2c6293196a0d32961cc: Status 404 returned error can't find the container with id a9c8e90858bf2e467e1ca745717616f43e9851925322c2c6293196a0d32961cc Jul 07 18:51:02.673365 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:02.673330 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-98vpd\"" Jul 07 18:51:02.681531 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:02.681506 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs" Jul 07 18:51:02.826712 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:02.826683 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs"] Jul 07 18:51:02.829630 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:51:02.829599 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64e169d3_cc4a_46a0_bfe6_1fb28c8deeae.slice/crio-c2964b3bc2c36c38149f96e10923a646e7f8da52c97c6c121cf957502a672e41 WatchSource:0}: Error finding container c2964b3bc2c36c38149f96e10923a646e7f8da52c97c6c121cf957502a672e41: Status 404 returned error can't find the container with id c2964b3bc2c36c38149f96e10923a646e7f8da52c97c6c121cf957502a672e41 Jul 07 18:51:02.979728 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:02.979687 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ffm5c" event={"ID":"570bed30-5e16-475c-9cc4-4e5af148c2e5","Type":"ContainerStarted","Data":"5a799328701b22db65fab4bed5e8bde90cd09d66267b93087d5cd6b3f49ce5a5"} Jul 07 18:51:02.981556 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:02.981530 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mkg6r" event={"ID":"88340235-0f7e-4e90-bf52-86b3f04aa9ff","Type":"ContainerStarted","Data":"965903921dd5d171f551d3b9ceecfee2d0588d06447dcf1e08bf4661226d6781"} Jul 07 18:51:02.984106 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:02.984074 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lksfn" event={"ID":"5a4777b9-2b1e-4987-9073-426919e6919c","Type":"ContainerStarted","Data":"efb8dee5a6d57f5733b896f36137189a1062e0521fb2f167dc34b73e85fad927"} Jul 07 18:51:02.985244 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:02.985224 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs" event={"ID":"64e169d3-cc4a-46a0-bfe6-1fb28c8deeae","Type":"ContainerStarted","Data":"c2964b3bc2c36c38149f96e10923a646e7f8da52c97c6c121cf957502a672e41"} Jul 07 18:51:02.986517 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:02.986495 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-cxwq7" event={"ID":"9250496d-2e44-489b-b3c0-b6c3121077a3","Type":"ContainerStarted","Data":"a9c8e90858bf2e467e1ca745717616f43e9851925322c2c6293196a0d32961cc"} Jul 07 18:51:02.987732 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:02.987708 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5667689847-7pjmn" event={"ID":"9873de59-358e-420f-ab91-dcd142bdc45b","Type":"ContainerStarted","Data":"f66947ab44aba047187c9bb6d4581e123f750d76a08377c31c2c4f5f55b376fc"} Jul 07 18:51:02.988774 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:02.988746 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b57q9" event={"ID":"fa1690c9-bef6-43d0-9aff-2359f4dfc39f","Type":"ContainerStarted","Data":"a2ac6096bdac2029ded0baed370940ee14a2f74c0f2f7e1a48c08dfc84099a7f"} Jul 07 18:51:02.996183 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:02.996137 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ffm5c" podStartSLOduration=69.062495411 podStartE2EDuration="1m14.996125929s" totalImagesPullingTime="5.933630518s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:49:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:02.995675799 +0000 UTC m=+74.965311831" watchObservedRunningTime="2026-07-07 18:51:02.996125929 +0000 UTC m=+74.965761946" Jul 07 18:51:03.014528 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:03.014472 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lksfn" podStartSLOduration=18.095702498 podStartE2EDuration="28.014454192s" totalImagesPullingTime="9.918751694s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 18:50:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:03.013832919 +0000 UTC m=+74.983468936" watchObservedRunningTime="2026-07-07 18:51:03.014454192 +0000 UTC m=+74.984090213" Jul 07 18:51:07.007538 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:07.007268 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs" event={"ID":"64e169d3-cc4a-46a0-bfe6-1fb28c8deeae","Type":"ContainerStarted","Data":"79e7ad1eea52b1901f271a17919cfd1c1cc0d935982d14cf636018bee7558017"} Jul 07 18:51:07.010448 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:07.010360 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-cxwq7" event={"ID":"9250496d-2e44-489b-b3c0-b6c3121077a3","Type":"ContainerStarted","Data":"f05341d6b0efcfb5265407fb9bb86ba9803c46fe59559e25ab15817cb2711dd6"} Jul 07 18:51:07.011981 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:07.011963 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5667689847-7pjmn" event={"ID":"9873de59-358e-420f-ab91-dcd142bdc45b","Type":"ContainerStarted","Data":"e81c3d64234f7420509eea115470c15880f4027bf7e569251f259130f0570f8d"} Jul 07 18:51:07.012483 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:07.012462 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5667689847-7pjmn" Jul 07 18:51:07.013844 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:07.013819 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b57q9" event={"ID":"fa1690c9-bef6-43d0-9aff-2359f4dfc39f","Type":"ContainerStarted","Data":"dce4ae84cccc578a70ed40877e8af447dea3822399f94e347381f97569afcbc6"} Jul 07 18:51:07.013937 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:07.013896 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5667689847-7pjmn" Jul 07 18:51:07.027389 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:07.027344 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-m8fgs" podStartSLOduration=33.035274051 podStartE2EDuration="37.027331598s" totalImagesPullingTime="3.992057547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:07.026300654 +0000 UTC m=+78.995936672" watchObservedRunningTime="2026-07-07 18:51:07.027331598 +0000 UTC m=+78.996967628" Jul 07 18:51:07.045093 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:07.045029 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-cxwq7" podStartSLOduration=32.822292608 podStartE2EDuration="37.045009436s" totalImagesPullingTime="4.222716828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:07.04438463 +0000 UTC m=+79.014020647" watchObservedRunningTime="2026-07-07 18:51:07.045009436 +0000 UTC m=+79.014645454" Jul 07 18:51:07.071106 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:07.071017 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5667689847-7pjmn" podStartSLOduration=4.267780857 podStartE2EDuration="9.070999791s" totalImagesPullingTime="4.803218934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:07.070358139 +0000 UTC m=+79.039994157" watchObservedRunningTime="2026-07-07 18:51:07.070999791 +0000 UTC m=+79.040635808" Jul 07 18:51:07.401266 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:07.401162 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bsgkt"] Jul 07 18:51:07.423532 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:07.423502 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bsgkt"] Jul 07 18:51:07.423692 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:07.423606 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bsgkt" Jul 07 18:51:07.426210 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:07.426168 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 07 18:51:07.426328 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:07.426264 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-lr2jj\"" Jul 07 18:51:07.558936 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:07.558907 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a70abc33-74af-46f8-8f77-e1f0b3065270-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-bsgkt\" (UID: \"a70abc33-74af-46f8-8f77-e1f0b3065270\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bsgkt" Jul 07 18:51:07.659693 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:07.659604 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a70abc33-74af-46f8-8f77-e1f0b3065270-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-bsgkt\" (UID: \"a70abc33-74af-46f8-8f77-e1f0b3065270\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bsgkt" Jul 07 18:51:07.659857 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:51:07.659739 2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 07 18:51:07.659857 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:51:07.659811 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a70abc33-74af-46f8-8f77-e1f0b3065270-tls-certificates podName:a70abc33-74af-46f8-8f77-e1f0b3065270 nodeName:}" failed. No retries permitted until 2026-07-07 18:51:08.159788896 +0000 UTC m=+80.129424905 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/a70abc33-74af-46f8-8f77-e1f0b3065270-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-bsgkt" (UID: "a70abc33-74af-46f8-8f77-e1f0b3065270") : secret "prometheus-operator-admission-webhook-tls" not found Jul 07 18:51:08.018463 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:08.018432 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b57q9" event={"ID":"fa1690c9-bef6-43d0-9aff-2359f4dfc39f","Type":"ContainerStarted","Data":"3d0892ec0b8dbba834c232a6a279783a296c1eed00e83596ec8b937d1ecfd362"} Jul 07 18:51:08.040211 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:08.040151 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-b57q9" podStartSLOduration=36.242908207 podStartE2EDuration="41.040130644s" totalImagesPullingTime="4.797222437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:50:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:08.038741412 +0000 UTC m=+80.008377442" watchObservedRunningTime="2026-07-07 18:51:08.040130644 +0000 UTC m=+80.009766661" Jul 07 18:51:08.165503 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:08.165232 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a70abc33-74af-46f8-8f77-e1f0b3065270-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-bsgkt\" (UID: \"a70abc33-74af-46f8-8f77-e1f0b3065270\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bsgkt" Jul 07 18:51:08.165503 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:51:08.165348 2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 07 18:51:08.165503 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:51:08.165429 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a70abc33-74af-46f8-8f77-e1f0b3065270-tls-certificates podName:a70abc33-74af-46f8-8f77-e1f0b3065270 nodeName:}" failed. No retries permitted until 2026-07-07 18:51:09.165405812 +0000 UTC m=+81.135041827 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/a70abc33-74af-46f8-8f77-e1f0b3065270-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-bsgkt" (UID: "a70abc33-74af-46f8-8f77-e1f0b3065270") : secret "prometheus-operator-admission-webhook-tls" not found Jul 07 18:51:08.515503 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:08.515467 2564 patch_prober.go:28] interesting pod/image-registry-569b766d4c-sdn6q container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 07 18:51:08.515700 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:08.515519 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" podUID="fe234b89-9fa1-4515-a421-e5f3e4393eae" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 18:51:09.022529 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:09.022496 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-b57q9" Jul 07 18:51:09.173337 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:09.173287 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a70abc33-74af-46f8-8f77-e1f0b3065270-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-bsgkt\" (UID: \"a70abc33-74af-46f8-8f77-e1f0b3065270\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bsgkt" Jul 07 18:51:09.175677 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:09.175653 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/a70abc33-74af-46f8-8f77-e1f0b3065270-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-bsgkt\" (UID: \"a70abc33-74af-46f8-8f77-e1f0b3065270\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bsgkt" Jul 07 18:51:09.232764 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:09.232727 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bsgkt" Jul 07 18:51:09.383883 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:09.383851 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bsgkt"] Jul 07 18:51:09.387613 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:51:09.387584 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda70abc33_74af_46f8_8f77_e1f0b3065270.slice/crio-8137a1404812d71fda95e2179773434aa20b1c15205b35e8e32543e955eb623e WatchSource:0}: Error finding container 8137a1404812d71fda95e2179773434aa20b1c15205b35e8e32543e955eb623e: Status 404 returned error can't find the container with id 8137a1404812d71fda95e2179773434aa20b1c15205b35e8e32543e955eb623e Jul 07 18:51:10.027682 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:10.027645 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bsgkt" event={"ID":"a70abc33-74af-46f8-8f77-e1f0b3065270","Type":"ContainerStarted","Data":"8137a1404812d71fda95e2179773434aa20b1c15205b35e8e32543e955eb623e"} Jul 07 18:51:10.935659 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:10.935627 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b54b55556-kmbdb"] Jul 07 18:51:10.938620 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:10.938604 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:10.940932 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:10.940910 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 18:51:10.941161 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:10.941144 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 18:51:10.941899 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:10.941884 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 18:51:10.941965 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:10.941908 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 18:51:10.942132 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:10.942112 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 18:51:10.942226 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:10.942133 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 18:51:10.942226 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:10.942142 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 18:51:10.942347 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:10.942275 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zh69j\"" Jul 07 18:51:10.953933 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:10.953909 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b54b55556-kmbdb"] Jul 07 18:51:11.033019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.032984 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bsgkt" event={"ID":"a70abc33-74af-46f8-8f77-e1f0b3065270","Type":"ContainerStarted","Data":"2680fe719ae1384f21a4cdad25720e37eed46611b56f8950168c21606cc7b0af"} Jul 07 18:51:11.033492 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.033141 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bsgkt" Jul 07 18:51:11.037757 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.037735 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bsgkt" Jul 07 18:51:11.051309 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.051272 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-bsgkt" podStartSLOduration=3.072478978 podStartE2EDuration="4.051258156s" totalImagesPullingTime="978.779178ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:11.050537051 +0000 UTC m=+83.020173090" watchObservedRunningTime="2026-07-07 18:51:11.051258156 +0000 UTC m=+83.020894172" Jul 07 18:51:11.089246 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.089214 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe8ffd85-f400-45e4-b88d-457e745d9132-oauth-serving-cert\") pod \"console-5b54b55556-kmbdb\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:11.089410 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.089252 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe8ffd85-f400-45e4-b88d-457e745d9132-console-serving-cert\") pod \"console-5b54b55556-kmbdb\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:11.089410 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.089273 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe8ffd85-f400-45e4-b88d-457e745d9132-console-oauth-config\") pod \"console-5b54b55556-kmbdb\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:11.089410 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.089299 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe8ffd85-f400-45e4-b88d-457e745d9132-console-config\") pod \"console-5b54b55556-kmbdb\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:11.089410 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.089387 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe8ffd85-f400-45e4-b88d-457e745d9132-service-ca\") pod \"console-5b54b55556-kmbdb\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:11.089578 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.089425 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmrg4\" (UniqueName: \"kubernetes.io/projected/fe8ffd85-f400-45e4-b88d-457e745d9132-kube-api-access-dmrg4\") pod \"console-5b54b55556-kmbdb\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:11.190484 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.190457 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe8ffd85-f400-45e4-b88d-457e745d9132-oauth-serving-cert\") pod \"console-5b54b55556-kmbdb\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:11.190652 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.190491 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe8ffd85-f400-45e4-b88d-457e745d9132-console-serving-cert\") pod \"console-5b54b55556-kmbdb\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:11.190652 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.190508 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe8ffd85-f400-45e4-b88d-457e745d9132-console-oauth-config\") pod \"console-5b54b55556-kmbdb\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:11.190652 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.190526 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe8ffd85-f400-45e4-b88d-457e745d9132-console-config\") pod \"console-5b54b55556-kmbdb\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:11.190652 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.190551 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe8ffd85-f400-45e4-b88d-457e745d9132-service-ca\") pod \"console-5b54b55556-kmbdb\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:11.190652 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.190576 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmrg4\" (UniqueName: \"kubernetes.io/projected/fe8ffd85-f400-45e4-b88d-457e745d9132-kube-api-access-dmrg4\") pod \"console-5b54b55556-kmbdb\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:11.191785 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.191574 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe8ffd85-f400-45e4-b88d-457e745d9132-oauth-serving-cert\") pod \"console-5b54b55556-kmbdb\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:11.191785 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.191598 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe8ffd85-f400-45e4-b88d-457e745d9132-service-ca\") pod \"console-5b54b55556-kmbdb\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:11.191785 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.191574 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe8ffd85-f400-45e4-b88d-457e745d9132-console-config\") pod \"console-5b54b55556-kmbdb\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:11.194216 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.194163 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe8ffd85-f400-45e4-b88d-457e745d9132-console-serving-cert\") pod \"console-5b54b55556-kmbdb\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:11.195533 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.195509 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe8ffd85-f400-45e4-b88d-457e745d9132-console-oauth-config\") pod \"console-5b54b55556-kmbdb\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:11.200555 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.200528 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmrg4\" (UniqueName: \"kubernetes.io/projected/fe8ffd85-f400-45e4-b88d-457e745d9132-kube-api-access-dmrg4\") pod \"console-5b54b55556-kmbdb\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:11.247431 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.247409 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:11.376057 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.376031 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b54b55556-kmbdb"] Jul 07 18:51:11.378927 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:51:11.378897 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe8ffd85_f400_45e4_b88d_457e745d9132.slice/crio-80e387ae880ed0018789eb8639df6a96d5bb385d2d55d9443fbccc1424840a62 WatchSource:0}: Error finding container 80e387ae880ed0018789eb8639df6a96d5bb385d2d55d9443fbccc1424840a62: Status 404 returned error can't find the container with id 80e387ae880ed0018789eb8639df6a96d5bb385d2d55d9443fbccc1424840a62 Jul 07 18:51:11.466768 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.466704 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd"] Jul 07 18:51:11.470952 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.470935 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd" Jul 07 18:51:11.473533 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.473514 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 18:51:11.473632 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.473567 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 07 18:51:11.473632 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.473587 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-jdb5x\"" Jul 07 18:51:11.473632 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.473597 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 07 18:51:11.482932 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.482915 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd"] Jul 07 18:51:11.593458 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.593433 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/05c43c32-65c0-402e-80b6-d0da89b9a7a0-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-hfvdd\" (UID: \"05c43c32-65c0-402e-80b6-d0da89b9a7a0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd" Jul 07 18:51:11.593458 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.593460 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05c43c32-65c0-402e-80b6-d0da89b9a7a0-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-hfvdd\" (UID: \"05c43c32-65c0-402e-80b6-d0da89b9a7a0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd" Jul 07 18:51:11.593625 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.593483 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp2bk\" (UniqueName: \"kubernetes.io/projected/05c43c32-65c0-402e-80b6-d0da89b9a7a0-kube-api-access-fp2bk\") pod \"prometheus-operator-567f7b8b7c-hfvdd\" (UID: \"05c43c32-65c0-402e-80b6-d0da89b9a7a0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd" Jul 07 18:51:11.593625 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.593515 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05c43c32-65c0-402e-80b6-d0da89b9a7a0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-hfvdd\" (UID: \"05c43c32-65c0-402e-80b6-d0da89b9a7a0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd" Jul 07 18:51:11.694181 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.694155 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05c43c32-65c0-402e-80b6-d0da89b9a7a0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-hfvdd\" (UID: \"05c43c32-65c0-402e-80b6-d0da89b9a7a0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd" Jul 07 18:51:11.694328 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.694228 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/05c43c32-65c0-402e-80b6-d0da89b9a7a0-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-hfvdd\" (UID: \"05c43c32-65c0-402e-80b6-d0da89b9a7a0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd" Jul 07 18:51:11.694328 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:51:11.694302 2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 07 18:51:11.694446 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.694332 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05c43c32-65c0-402e-80b6-d0da89b9a7a0-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-hfvdd\" (UID: \"05c43c32-65c0-402e-80b6-d0da89b9a7a0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd" Jul 07 18:51:11.694446 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:51:11.694370 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05c43c32-65c0-402e-80b6-d0da89b9a7a0-prometheus-operator-tls podName:05c43c32-65c0-402e-80b6-d0da89b9a7a0 nodeName:}" failed. No retries permitted until 2026-07-07 18:51:12.194353868 +0000 UTC m=+84.163989863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/05c43c32-65c0-402e-80b6-d0da89b9a7a0-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-hfvdd" (UID: "05c43c32-65c0-402e-80b6-d0da89b9a7a0") : secret "prometheus-operator-tls" not found Jul 07 18:51:11.694446 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.694392 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp2bk\" (UniqueName: \"kubernetes.io/projected/05c43c32-65c0-402e-80b6-d0da89b9a7a0-kube-api-access-fp2bk\") pod \"prometheus-operator-567f7b8b7c-hfvdd\" (UID: \"05c43c32-65c0-402e-80b6-d0da89b9a7a0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd" Jul 07 18:51:11.694993 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.694971 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/05c43c32-65c0-402e-80b6-d0da89b9a7a0-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-hfvdd\" (UID: \"05c43c32-65c0-402e-80b6-d0da89b9a7a0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd" Jul 07 18:51:11.696352 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.696334 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/05c43c32-65c0-402e-80b6-d0da89b9a7a0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-hfvdd\" (UID: \"05c43c32-65c0-402e-80b6-d0da89b9a7a0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd" Jul 07 18:51:11.703617 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:11.703599 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp2bk\" (UniqueName: \"kubernetes.io/projected/05c43c32-65c0-402e-80b6-d0da89b9a7a0-kube-api-access-fp2bk\") pod \"prometheus-operator-567f7b8b7c-hfvdd\" (UID: \"05c43c32-65c0-402e-80b6-d0da89b9a7a0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd" Jul 07 18:51:12.036929 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:12.036894 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b54b55556-kmbdb" event={"ID":"fe8ffd85-f400-45e4-b88d-457e745d9132","Type":"ContainerStarted","Data":"80e387ae880ed0018789eb8639df6a96d5bb385d2d55d9443fbccc1424840a62"} Jul 07 18:51:12.198398 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:12.198363 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/05c43c32-65c0-402e-80b6-d0da89b9a7a0-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-hfvdd\" (UID: \"05c43c32-65c0-402e-80b6-d0da89b9a7a0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd" Jul 07 18:51:12.198575 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:51:12.198499 2564 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 07 18:51:12.198575 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:51:12.198567 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05c43c32-65c0-402e-80b6-d0da89b9a7a0-prometheus-operator-tls podName:05c43c32-65c0-402e-80b6-d0da89b9a7a0 nodeName:}" failed. No retries permitted until 2026-07-07 18:51:13.19854809 +0000 UTC m=+85.168184099 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/05c43c32-65c0-402e-80b6-d0da89b9a7a0-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-hfvdd" (UID: "05c43c32-65c0-402e-80b6-d0da89b9a7a0") : secret "prometheus-operator-tls" not found Jul 07 18:51:13.207188 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:13.207146 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/05c43c32-65c0-402e-80b6-d0da89b9a7a0-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-hfvdd\" (UID: \"05c43c32-65c0-402e-80b6-d0da89b9a7a0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd" Jul 07 18:51:13.210294 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:13.210265 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/05c43c32-65c0-402e-80b6-d0da89b9a7a0-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-hfvdd\" (UID: \"05c43c32-65c0-402e-80b6-d0da89b9a7a0\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd" Jul 07 18:51:13.280467 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:13.280437 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd" Jul 07 18:51:13.416387 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:13.416328 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd"] Jul 07 18:51:13.419260 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:51:13.419224 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05c43c32_65c0_402e_80b6_d0da89b9a7a0.slice/crio-545a3f8f5afd26148088c61b41c595126b2c54123f2e53ba99723eee31e20a6e WatchSource:0}: Error finding container 545a3f8f5afd26148088c61b41c595126b2c54123f2e53ba99723eee31e20a6e: Status 404 returned error can't find the container with id 545a3f8f5afd26148088c61b41c595126b2c54123f2e53ba99723eee31e20a6e Jul 07 18:51:14.046544 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:14.046508 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd" event={"ID":"05c43c32-65c0-402e-80b6-d0da89b9a7a0","Type":"ContainerStarted","Data":"545a3f8f5afd26148088c61b41c595126b2c54123f2e53ba99723eee31e20a6e"} Jul 07 18:51:15.052637 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:15.052532 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b54b55556-kmbdb" event={"ID":"fe8ffd85-f400-45e4-b88d-457e745d9132","Type":"ContainerStarted","Data":"8c391c48cbbc88161399a1e278167870c0ddc653c98c098638e6230f6068e737"} Jul 07 18:51:15.079941 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:15.079897 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b54b55556-kmbdb" podStartSLOduration=1.725846063 podStartE2EDuration="5.079876269s" totalImagesPullingTime="3.354030206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:51:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:15.079660443 +0000 UTC m=+87.049296482" watchObservedRunningTime="2026-07-07 18:51:15.079876269 +0000 UTC m=+87.049512288" Jul 07 18:51:16.029537 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:16.029512 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-b57q9" Jul 07 18:51:16.058427 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:16.058390 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd" event={"ID":"05c43c32-65c0-402e-80b6-d0da89b9a7a0","Type":"ContainerStarted","Data":"cd5e0860c771b1c04c95d406807256ac987b8aa9a5f9dff98dc9f04b3d8981cc"} Jul 07 18:51:16.058846 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:16.058472 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd" event={"ID":"05c43c32-65c0-402e-80b6-d0da89b9a7a0","Type":"ContainerStarted","Data":"520d85e3c9cd4360e536229de3e5196e090efa9463b5c786d78716c8a8daa589"} Jul 07 18:51:16.081909 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:16.081846 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-hfvdd" podStartSLOduration=3.058276827 podStartE2EDuration="5.081827741s" totalImagesPullingTime="2.023550914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:51:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:16.081151901 +0000 UTC m=+88.050787946" watchObservedRunningTime="2026-07-07 18:51:16.081827741 +0000 UTC m=+88.051463759" Jul 07 18:51:17.897267 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:17.897238 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47"] Jul 07 18:51:17.900448 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:17.900432 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47" Jul 07 18:51:17.902697 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:17.902679 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 07 18:51:17.902817 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:17.902801 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 07 18:51:17.903712 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:17.903698 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-f2rfq\"" Jul 07 18:51:17.914950 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:17.914929 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47"] Jul 07 18:51:17.922898 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:17.922878 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-4s89j"] Jul 07 18:51:17.926094 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:17.926077 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:17.928568 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:17.928536 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 18:51:17.928568 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:17.928544 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 18:51:17.928802 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:17.928789 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 18:51:17.928843 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:17.928825 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fqp2b\"" Jul 07 18:51:17.935337 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:17.935318 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn"] Jul 07 18:51:17.938391 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:17.938376 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" Jul 07 18:51:17.940784 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:17.940767 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 07 18:51:17.940784 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:17.940782 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 07 18:51:17.940919 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:17.940820 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-7t5bj\"" Jul 07 18:51:17.940919 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:17.940830 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 07 18:51:17.964677 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:17.964657 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn"] Jul 07 18:51:18.043671 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.043644 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2e5e2610-e42a-4689-990c-d681773d0d1e-node-exporter-wtmp\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.043671 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.043672 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhxhj\" (UniqueName: \"kubernetes.io/projected/7566d1fb-829e-4376-a8d6-fc2239d8e32c-kube-api-access-qhxhj\") pod \"kube-state-metrics-7c9dbb8b7d-rcvxn\" (UID: \"7566d1fb-829e-4376-a8d6-fc2239d8e32c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" Jul 07 18:51:18.043830 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.043692 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e5e2610-e42a-4689-990c-d681773d0d1e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.043830 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.043718 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7566d1fb-829e-4376-a8d6-fc2239d8e32c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-rcvxn\" (UID: \"7566d1fb-829e-4376-a8d6-fc2239d8e32c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" Jul 07 18:51:18.043830 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.043757 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2e5e2610-e42a-4689-990c-d681773d0d1e-node-exporter-accelerators-collector-config\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.043830 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.043779 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e8311d0-5139-405b-9fcb-709fceb273db-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-nkx47\" (UID: \"7e8311d0-5139-405b-9fcb-709fceb273db\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47" Jul 07 18:51:18.043830 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.043812 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r57v7\" (UniqueName: \"kubernetes.io/projected/2e5e2610-e42a-4689-990c-d681773d0d1e-kube-api-access-r57v7\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.044030 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.043835 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7566d1fb-829e-4376-a8d6-fc2239d8e32c-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-rcvxn\" (UID: \"7566d1fb-829e-4376-a8d6-fc2239d8e32c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" Jul 07 18:51:18.044030 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.043866 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7566d1fb-829e-4376-a8d6-fc2239d8e32c-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-rcvxn\" (UID: \"7566d1fb-829e-4376-a8d6-fc2239d8e32c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" Jul 07 18:51:18.044030 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.043882 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2e5e2610-e42a-4689-990c-d681773d0d1e-node-exporter-textfile\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.044030 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.043908 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e5e2610-e42a-4689-990c-d681773d0d1e-sys\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.044030 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.043931 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2e5e2610-e42a-4689-990c-d681773d0d1e-node-exporter-tls\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.044030 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.043971 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e8311d0-5139-405b-9fcb-709fceb273db-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-nkx47\" (UID: \"7e8311d0-5139-405b-9fcb-709fceb273db\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47" Jul 07 18:51:18.044030 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.043993 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e5e2610-e42a-4689-990c-d681773d0d1e-metrics-client-ca\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.044030 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.044019 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7566d1fb-829e-4376-a8d6-fc2239d8e32c-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-rcvxn\" (UID: \"7566d1fb-829e-4376-a8d6-fc2239d8e32c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" Jul 07 18:51:18.044319 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.044034 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e8311d0-5139-405b-9fcb-709fceb273db-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-nkx47\" (UID: \"7e8311d0-5139-405b-9fcb-709fceb273db\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47" Jul 07 18:51:18.044319 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.044056 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2e5e2610-e42a-4689-990c-d681773d0d1e-root\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.044319 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.044114 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7566d1fb-829e-4376-a8d6-fc2239d8e32c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-rcvxn\" (UID: \"7566d1fb-829e-4376-a8d6-fc2239d8e32c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" Jul 07 18:51:18.044319 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.044152 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s65l5\" (UniqueName: \"kubernetes.io/projected/7e8311d0-5139-405b-9fcb-709fceb273db-kube-api-access-s65l5\") pod \"openshift-state-metrics-54cf96dfcf-nkx47\" (UID: \"7e8311d0-5139-405b-9fcb-709fceb273db\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47" Jul 07 18:51:18.145427 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.145403 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e5e2610-e42a-4689-990c-d681773d0d1e-sys\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.145427 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.145431 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2e5e2610-e42a-4689-990c-d681773d0d1e-node-exporter-tls\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.145589 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.145454 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e8311d0-5139-405b-9fcb-709fceb273db-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-nkx47\" (UID: \"7e8311d0-5139-405b-9fcb-709fceb273db\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47" Jul 07 18:51:18.145589 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.145473 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e5e2610-e42a-4689-990c-d681773d0d1e-metrics-client-ca\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.145589 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.145501 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7566d1fb-829e-4376-a8d6-fc2239d8e32c-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-rcvxn\" (UID: \"7566d1fb-829e-4376-a8d6-fc2239d8e32c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" Jul 07 18:51:18.145589 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.145521 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2e5e2610-e42a-4689-990c-d681773d0d1e-sys\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.145589 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:51:18.145567 2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 07 18:51:18.145825 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.145607 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e8311d0-5139-405b-9fcb-709fceb273db-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-nkx47\" (UID: \"7e8311d0-5139-405b-9fcb-709fceb273db\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47" Jul 07 18:51:18.145825 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:51:18.145633 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e5e2610-e42a-4689-990c-d681773d0d1e-node-exporter-tls podName:2e5e2610-e42a-4689-990c-d681773d0d1e nodeName:}" failed. No retries permitted until 2026-07-07 18:51:18.645614309 +0000 UTC m=+90.615250308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2e5e2610-e42a-4689-990c-d681773d0d1e-node-exporter-tls") pod "node-exporter-4s89j" (UID: "2e5e2610-e42a-4689-990c-d681773d0d1e") : secret "node-exporter-tls" not found Jul 07 18:51:18.145825 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.145666 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2e5e2610-e42a-4689-990c-d681773d0d1e-root\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.145825 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.145699 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7566d1fb-829e-4376-a8d6-fc2239d8e32c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-rcvxn\" (UID: \"7566d1fb-829e-4376-a8d6-fc2239d8e32c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" Jul 07 18:51:18.145825 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.145736 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s65l5\" (UniqueName: \"kubernetes.io/projected/7e8311d0-5139-405b-9fcb-709fceb273db-kube-api-access-s65l5\") pod \"openshift-state-metrics-54cf96dfcf-nkx47\" (UID: \"7e8311d0-5139-405b-9fcb-709fceb273db\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47" Jul 07 18:51:18.145825 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.145747 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2e5e2610-e42a-4689-990c-d681773d0d1e-root\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.145825 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.145766 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2e5e2610-e42a-4689-990c-d681773d0d1e-node-exporter-wtmp\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.145825 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.145789 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhxhj\" (UniqueName: \"kubernetes.io/projected/7566d1fb-829e-4376-a8d6-fc2239d8e32c-kube-api-access-qhxhj\") pod \"kube-state-metrics-7c9dbb8b7d-rcvxn\" (UID: \"7566d1fb-829e-4376-a8d6-fc2239d8e32c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" Jul 07 18:51:18.145825 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.145813 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e5e2610-e42a-4689-990c-d681773d0d1e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.146288 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.145860 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7566d1fb-829e-4376-a8d6-fc2239d8e32c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-rcvxn\" (UID: \"7566d1fb-829e-4376-a8d6-fc2239d8e32c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" Jul 07 18:51:18.146288 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.145981 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2e5e2610-e42a-4689-990c-d681773d0d1e-node-exporter-wtmp\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.146288 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.146021 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2e5e2610-e42a-4689-990c-d681773d0d1e-node-exporter-accelerators-collector-config\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.146288 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.146060 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e8311d0-5139-405b-9fcb-709fceb273db-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-nkx47\" (UID: \"7e8311d0-5139-405b-9fcb-709fceb273db\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47" Jul 07 18:51:18.146288 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.146099 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r57v7\" (UniqueName: \"kubernetes.io/projected/2e5e2610-e42a-4689-990c-d681773d0d1e-kube-api-access-r57v7\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.146288 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.146127 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7566d1fb-829e-4376-a8d6-fc2239d8e32c-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-rcvxn\" (UID: \"7566d1fb-829e-4376-a8d6-fc2239d8e32c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" Jul 07 18:51:18.146288 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.146147 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e5e2610-e42a-4689-990c-d681773d0d1e-metrics-client-ca\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.146288 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.146158 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7566d1fb-829e-4376-a8d6-fc2239d8e32c-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-rcvxn\" (UID: \"7566d1fb-829e-4376-a8d6-fc2239d8e32c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" Jul 07 18:51:18.146288 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.146187 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2e5e2610-e42a-4689-990c-d681773d0d1e-node-exporter-textfile\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.146288 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.146253 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7566d1fb-829e-4376-a8d6-fc2239d8e32c-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-rcvxn\" (UID: \"7566d1fb-829e-4376-a8d6-fc2239d8e32c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" Jul 07 18:51:18.146762 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.146305 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e8311d0-5139-405b-9fcb-709fceb273db-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-nkx47\" (UID: \"7e8311d0-5139-405b-9fcb-709fceb273db\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47" Jul 07 18:51:18.146762 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.146471 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2e5e2610-e42a-4689-990c-d681773d0d1e-node-exporter-textfile\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.146762 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.146533 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/7566d1fb-829e-4376-a8d6-fc2239d8e32c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-rcvxn\" (UID: \"7566d1fb-829e-4376-a8d6-fc2239d8e32c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" Jul 07 18:51:18.146762 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.146604 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/7566d1fb-829e-4376-a8d6-fc2239d8e32c-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-rcvxn\" (UID: \"7566d1fb-829e-4376-a8d6-fc2239d8e32c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" Jul 07 18:51:18.147157 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.147138 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2e5e2610-e42a-4689-990c-d681773d0d1e-node-exporter-accelerators-collector-config\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.148720 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.148667 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7566d1fb-829e-4376-a8d6-fc2239d8e32c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-rcvxn\" (UID: \"7566d1fb-829e-4376-a8d6-fc2239d8e32c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" Jul 07 18:51:18.148720 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.148667 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e8311d0-5139-405b-9fcb-709fceb273db-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-nkx47\" (UID: \"7e8311d0-5139-405b-9fcb-709fceb273db\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47" Jul 07 18:51:18.148917 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.148892 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7566d1fb-829e-4376-a8d6-fc2239d8e32c-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-rcvxn\" (UID: \"7566d1fb-829e-4376-a8d6-fc2239d8e32c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" Jul 07 18:51:18.148997 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.148976 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7e8311d0-5139-405b-9fcb-709fceb273db-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-nkx47\" (UID: \"7e8311d0-5139-405b-9fcb-709fceb273db\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47" Jul 07 18:51:18.149067 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.149050 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e5e2610-e42a-4689-990c-d681773d0d1e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.154626 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.154607 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhxhj\" (UniqueName: \"kubernetes.io/projected/7566d1fb-829e-4376-a8d6-fc2239d8e32c-kube-api-access-qhxhj\") pod \"kube-state-metrics-7c9dbb8b7d-rcvxn\" (UID: \"7566d1fb-829e-4376-a8d6-fc2239d8e32c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" Jul 07 18:51:18.155803 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.155786 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r57v7\" (UniqueName: \"kubernetes.io/projected/2e5e2610-e42a-4689-990c-d681773d0d1e-kube-api-access-r57v7\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.156189 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.156160 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s65l5\" (UniqueName: \"kubernetes.io/projected/7e8311d0-5139-405b-9fcb-709fceb273db-kube-api-access-s65l5\") pod \"openshift-state-metrics-54cf96dfcf-nkx47\" (UID: \"7e8311d0-5139-405b-9fcb-709fceb273db\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47" Jul 07 18:51:18.209188 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.209165 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47" Jul 07 18:51:18.246657 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.246630 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" Jul 07 18:51:18.350739 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.349515 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47"] Jul 07 18:51:18.352896 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:51:18.352863 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e8311d0_5139_405b_9fcb_709fceb273db.slice/crio-116ce1aa48df75142790e64a820ebfa5beeaa54f2500e93f783bbff13ed87d15 WatchSource:0}: Error finding container 116ce1aa48df75142790e64a820ebfa5beeaa54f2500e93f783bbff13ed87d15: Status 404 returned error can't find the container with id 116ce1aa48df75142790e64a820ebfa5beeaa54f2500e93f783bbff13ed87d15 Jul 07 18:51:18.399993 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.399876 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn"] Jul 07 18:51:18.402727 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:51:18.402708 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7566d1fb_829e_4376_a8d6_fc2239d8e32c.slice/crio-055d8019aa0cab4934a72d726a2d8ef5eae4aadee5e0cfaddddc92bdad125d47 WatchSource:0}: Error finding container 055d8019aa0cab4934a72d726a2d8ef5eae4aadee5e0cfaddddc92bdad125d47: Status 404 returned error can't find the container with id 055d8019aa0cab4934a72d726a2d8ef5eae4aadee5e0cfaddddc92bdad125d47 Jul 07 18:51:18.514946 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.514925 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:51:18.650880 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.650808 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2e5e2610-e42a-4689-990c-d681773d0d1e-node-exporter-tls\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.652901 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.652882 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2e5e2610-e42a-4689-990c-d681773d0d1e-node-exporter-tls\") pod \"node-exporter-4s89j\" (UID: \"2e5e2610-e42a-4689-990c-d681773d0d1e\") " pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.835415 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:18.835383 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4s89j" Jul 07 18:51:18.847713 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:51:18.847687 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e5e2610_e42a_4689_990c_d681773d0d1e.slice/crio-81eab9446903488e2cd10cd8cecba1684de0fb6c50e8504abda1fd0fef65d29e WatchSource:0}: Error finding container 81eab9446903488e2cd10cd8cecba1684de0fb6c50e8504abda1fd0fef65d29e: Status 404 returned error can't find the container with id 81eab9446903488e2cd10cd8cecba1684de0fb6c50e8504abda1fd0fef65d29e Jul 07 18:51:19.070776 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:19.070739 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4s89j" event={"ID":"2e5e2610-e42a-4689-990c-d681773d0d1e","Type":"ContainerStarted","Data":"81eab9446903488e2cd10cd8cecba1684de0fb6c50e8504abda1fd0fef65d29e"} Jul 07 18:51:19.073185 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:19.073054 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47" event={"ID":"7e8311d0-5139-405b-9fcb-709fceb273db","Type":"ContainerStarted","Data":"702076a90be253251497bd8604bc139b7a8c4261af5b5b6b95c4ee7e269922f0"} Jul 07 18:51:19.073185 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:19.073116 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47" event={"ID":"7e8311d0-5139-405b-9fcb-709fceb273db","Type":"ContainerStarted","Data":"cd7366661b4a51025492f90030817276fd836c6ef9cd23c9caa4a2b0a5b58142"} Jul 07 18:51:19.073185 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:19.073148 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47" event={"ID":"7e8311d0-5139-405b-9fcb-709fceb273db","Type":"ContainerStarted","Data":"116ce1aa48df75142790e64a820ebfa5beeaa54f2500e93f783bbff13ed87d15"} Jul 07 18:51:19.075422 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:19.075397 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" event={"ID":"7566d1fb-829e-4376-a8d6-fc2239d8e32c","Type":"ContainerStarted","Data":"055d8019aa0cab4934a72d726a2d8ef5eae4aadee5e0cfaddddc92bdad125d47"} Jul 07 18:51:20.898590 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:20.898556 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-8488ddff78-wthfd"] Jul 07 18:51:20.902208 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:20.902178 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:20.904869 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:20.904839 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 07 18:51:20.904869 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:20.904840 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 07 18:51:20.905037 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:20.904873 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 07 18:51:20.905037 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:20.904876 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-5pp9h5a9qb0un\"" Jul 07 18:51:20.905037 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:20.904954 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-jss5v\"" Jul 07 18:51:20.905258 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:20.905236 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 07 18:51:20.905258 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:20.905257 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 07 18:51:20.918527 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:20.918504 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8488ddff78-wthfd"] Jul 07 18:51:21.075042 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.075012 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkn9q\" (UniqueName: \"kubernetes.io/projected/f3010063-bc1a-4d37-95fb-c27e7d9720e7-kube-api-access-bkn9q\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.075182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.075056 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f3010063-bc1a-4d37-95fb-c27e7d9720e7-secret-thanos-querier-tls\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.075182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.075102 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3010063-bc1a-4d37-95fb-c27e7d9720e7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.075182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.075127 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3010063-bc1a-4d37-95fb-c27e7d9720e7-metrics-client-ca\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.075182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.075176 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f3010063-bc1a-4d37-95fb-c27e7d9720e7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.075411 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.075214 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3010063-bc1a-4d37-95fb-c27e7d9720e7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.075411 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.075230 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3010063-bc1a-4d37-95fb-c27e7d9720e7-secret-grpc-tls\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.075411 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.075338 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f3010063-bc1a-4d37-95fb-c27e7d9720e7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.086065 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.086041 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47" event={"ID":"7e8311d0-5139-405b-9fcb-709fceb273db","Type":"ContainerStarted","Data":"7ce1a5fdfe86ff56f1df48a5bb294d6d4b55f956f55380f906e655f8a0f950c9"} Jul 07 18:51:21.088078 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.088057 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" event={"ID":"7566d1fb-829e-4376-a8d6-fc2239d8e32c","Type":"ContainerStarted","Data":"80051e464471754aab0001250359639758d711371d3b024f08c87d15dca752d6"} Jul 07 18:51:21.088172 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.088106 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" event={"ID":"7566d1fb-829e-4376-a8d6-fc2239d8e32c","Type":"ContainerStarted","Data":"059f7d103440223f9b09402368578f67abae53da58ee9bad431be40dbaa8be6f"} Jul 07 18:51:21.088172 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.088125 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" event={"ID":"7566d1fb-829e-4376-a8d6-fc2239d8e32c","Type":"ContainerStarted","Data":"3db3e924b40cc47d5547f852214503c90143c50ea32538915c1bae6027d69b85"} Jul 07 18:51:21.089463 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.089441 2564 generic.go:356] "Generic (PLEG): container finished" podID="2e5e2610-e42a-4689-990c-d681773d0d1e" containerID="2df5798260f2c4d3450ffe9870484068c72131d254e40c52cd7858208c724266" exitCode=0 Jul 07 18:51:21.089546 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.089490 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4s89j" event={"ID":"2e5e2610-e42a-4689-990c-d681773d0d1e","Type":"ContainerDied","Data":"2df5798260f2c4d3450ffe9870484068c72131d254e40c52cd7858208c724266"} Jul 07 18:51:21.111450 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.111406 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkx47" podStartSLOduration=2.422948338 podStartE2EDuration="4.111389579s" totalImagesPullingTime="1.688441241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:51:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:21.109599396 +0000 UTC m=+93.079235413" watchObservedRunningTime="2026-07-07 18:51:21.111389579 +0000 UTC m=+93.081025600" Jul 07 18:51:21.157606 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.157319 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-rcvxn" podStartSLOduration=2.387939574 podStartE2EDuration="4.15730093s" totalImagesPullingTime="1.769361356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:51:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:21.15684094 +0000 UTC m=+93.126476964" watchObservedRunningTime="2026-07-07 18:51:21.15730093 +0000 UTC m=+93.126936949" Jul 07 18:51:21.176147 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.176125 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f3010063-bc1a-4d37-95fb-c27e7d9720e7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.176259 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.176155 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3010063-bc1a-4d37-95fb-c27e7d9720e7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.176259 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.176181 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3010063-bc1a-4d37-95fb-c27e7d9720e7-secret-grpc-tls\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.176364 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.176266 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f3010063-bc1a-4d37-95fb-c27e7d9720e7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.176364 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.176327 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkn9q\" (UniqueName: \"kubernetes.io/projected/f3010063-bc1a-4d37-95fb-c27e7d9720e7-kube-api-access-bkn9q\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.176481 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.176378 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f3010063-bc1a-4d37-95fb-c27e7d9720e7-secret-thanos-querier-tls\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.176481 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.176433 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3010063-bc1a-4d37-95fb-c27e7d9720e7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.176481 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.176462 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3010063-bc1a-4d37-95fb-c27e7d9720e7-metrics-client-ca\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.177557 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.177530 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3010063-bc1a-4d37-95fb-c27e7d9720e7-metrics-client-ca\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.179314 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.179242 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3010063-bc1a-4d37-95fb-c27e7d9720e7-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.179855 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.179796 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3010063-bc1a-4d37-95fb-c27e7d9720e7-secret-grpc-tls\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.179957 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.179862 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f3010063-bc1a-4d37-95fb-c27e7d9720e7-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.180014 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.179969 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3010063-bc1a-4d37-95fb-c27e7d9720e7-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.180132 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.180110 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f3010063-bc1a-4d37-95fb-c27e7d9720e7-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.180167 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.180126 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f3010063-bc1a-4d37-95fb-c27e7d9720e7-secret-thanos-querier-tls\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.185064 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.185042 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkn9q\" (UniqueName: \"kubernetes.io/projected/f3010063-bc1a-4d37-95fb-c27e7d9720e7-kube-api-access-bkn9q\") pod \"thanos-querier-8488ddff78-wthfd\" (UID: \"f3010063-bc1a-4d37-95fb-c27e7d9720e7\") " pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.210859 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.210836 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:21.248439 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.248411 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:21.248566 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.248476 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:21.254020 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.253999 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:21.357892 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:21.357862 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8488ddff78-wthfd"] Jul 07 18:51:21.360911 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:51:21.360885 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3010063_bc1a_4d37_95fb_c27e7d9720e7.slice/crio-7537eb210c32544d9613f4dc71dacbafac62b224a00d7d8c7487616c30a5993a WatchSource:0}: Error finding container 7537eb210c32544d9613f4dc71dacbafac62b224a00d7d8c7487616c30a5993a: Status 404 returned error can't find the container with id 7537eb210c32544d9613f4dc71dacbafac62b224a00d7d8c7487616c30a5993a Jul 07 18:51:22.094183 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:22.094146 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" event={"ID":"f3010063-bc1a-4d37-95fb-c27e7d9720e7","Type":"ContainerStarted","Data":"7537eb210c32544d9613f4dc71dacbafac62b224a00d7d8c7487616c30a5993a"} Jul 07 18:51:22.096230 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:22.096186 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4s89j" event={"ID":"2e5e2610-e42a-4689-990c-d681773d0d1e","Type":"ContainerStarted","Data":"2b5283a7db4337b5f3db59c19fe39dd83484e588d70f91d16f0b6d8932f464f6"} Jul 07 18:51:22.096367 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:22.096251 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4s89j" event={"ID":"2e5e2610-e42a-4689-990c-d681773d0d1e","Type":"ContainerStarted","Data":"e3f690391745bd3351e1478612eb94d723e3fb61130b9b892664c691933935ca"} Jul 07 18:51:22.100065 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:22.100041 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:51:22.141158 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:22.141113 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-4s89j" podStartSLOduration=3.727477956 podStartE2EDuration="5.141098759s" totalImagesPullingTime="1.326654758s" totalInitContainerRuntime="86.966045ms" isStatefulPod=true podCreationTimestamp="2026-07-07 18:51:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:22.120372878 +0000 UTC m=+94.090008930" watchObservedRunningTime="2026-07-07 18:51:22.141098759 +0000 UTC m=+94.110734775" Jul 07 18:51:23.536125 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:23.536073 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" podUID="fe234b89-9fa1-4515-a421-e5f3e4393eae" containerName="registry" containerID="cri-o://2aa85128bf8496d0ebcfb4ee773bd6acc4fc8ca31be48e0a4f4201b53a954516" gracePeriod=30 Jul 07 18:51:23.959127 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:23.959103 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:51:23.974009 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:23.972329 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d8759966d-pssgw"] Jul 07 18:51:23.974009 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:23.972755 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe234b89-9fa1-4515-a421-e5f3e4393eae" containerName="registry" Jul 07 18:51:23.974009 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:23.972771 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe234b89-9fa1-4515-a421-e5f3e4393eae" containerName="registry" Jul 07 18:51:23.974009 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:23.972856 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe234b89-9fa1-4515-a421-e5f3e4393eae" containerName="registry" Jul 07 18:51:23.978304 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:23.977312 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:23.984584 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:23.984555 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 18:51:23.988810 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:23.988538 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d8759966d-pssgw"] Jul 07 18:51:24.102275 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.102251 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-certificates\") pod \"fe234b89-9fa1-4515-a421-e5f3e4393eae\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " Jul 07 18:51:24.102397 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.102306 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-tls\") pod \"fe234b89-9fa1-4515-a421-e5f3e4393eae\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " Jul 07 18:51:24.102397 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.102354 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2tmz\" (UniqueName: \"kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-kube-api-access-f2tmz\") pod \"fe234b89-9fa1-4515-a421-e5f3e4393eae\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " Jul 07 18:51:24.102397 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.102384 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fe234b89-9fa1-4515-a421-e5f3e4393eae-installation-pull-secrets\") pod \"fe234b89-9fa1-4515-a421-e5f3e4393eae\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " Jul 07 18:51:24.102551 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.102423 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fe234b89-9fa1-4515-a421-e5f3e4393eae-image-registry-private-configuration\") pod \"fe234b89-9fa1-4515-a421-e5f3e4393eae\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " Jul 07 18:51:24.102551 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.102470 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-bound-sa-token\") pod \"fe234b89-9fa1-4515-a421-e5f3e4393eae\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " Jul 07 18:51:24.102551 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.102502 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fe234b89-9fa1-4515-a421-e5f3e4393eae-ca-trust-extracted\") pod \"fe234b89-9fa1-4515-a421-e5f3e4393eae\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " Jul 07 18:51:24.102551 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.102538 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fe234b89-9fa1-4515-a421-e5f3e4393eae-trusted-ca\") pod \"fe234b89-9fa1-4515-a421-e5f3e4393eae\" (UID: \"fe234b89-9fa1-4515-a421-e5f3e4393eae\") " Jul 07 18:51:24.102728 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.102652 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frs2n\" (UniqueName: \"kubernetes.io/projected/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-kube-api-access-frs2n\") pod \"console-5d8759966d-pssgw\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.102728 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.102720 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-console-config\") pod \"console-5d8759966d-pssgw\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.102876 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.102764 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-trusted-ca-bundle\") pod \"console-5d8759966d-pssgw\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.102876 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.102792 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-oauth-serving-cert\") pod \"console-5d8759966d-pssgw\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.102971 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.102877 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-console-serving-cert\") pod \"console-5d8759966d-pssgw\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.102971 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.102908 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "fe234b89-9fa1-4515-a421-e5f3e4393eae" (UID: "fe234b89-9fa1-4515-a421-e5f3e4393eae"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:51:24.102971 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.102917 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-console-oauth-config\") pod \"console-5d8759966d-pssgw\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.103126 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.102980 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-service-ca\") pod \"console-5d8759966d-pssgw\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.103126 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.103047 2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-certificates\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:51:24.103733 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.103697 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe234b89-9fa1-4515-a421-e5f3e4393eae-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "fe234b89-9fa1-4515-a421-e5f3e4393eae" (UID: "fe234b89-9fa1-4515-a421-e5f3e4393eae"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:51:24.105757 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.105359 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "fe234b89-9fa1-4515-a421-e5f3e4393eae" (UID: "fe234b89-9fa1-4515-a421-e5f3e4393eae"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:51:24.105757 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.105512 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "fe234b89-9fa1-4515-a421-e5f3e4393eae" (UID: "fe234b89-9fa1-4515-a421-e5f3e4393eae"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:51:24.105757 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.105593 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe234b89-9fa1-4515-a421-e5f3e4393eae-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "fe234b89-9fa1-4515-a421-e5f3e4393eae" (UID: "fe234b89-9fa1-4515-a421-e5f3e4393eae"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:51:24.105934 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.105762 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe234b89-9fa1-4515-a421-e5f3e4393eae-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "fe234b89-9fa1-4515-a421-e5f3e4393eae" (UID: "fe234b89-9fa1-4515-a421-e5f3e4393eae"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:51:24.106245 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.106224 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-kube-api-access-f2tmz" (OuterVolumeSpecName: "kube-api-access-f2tmz") pod "fe234b89-9fa1-4515-a421-e5f3e4393eae" (UID: "fe234b89-9fa1-4515-a421-e5f3e4393eae"). InnerVolumeSpecName "kube-api-access-f2tmz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:51:24.110557 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.110516 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" event={"ID":"f3010063-bc1a-4d37-95fb-c27e7d9720e7","Type":"ContainerStarted","Data":"0d289e7ab0365290ccd52a969329e6aec51a7cbafac38602255badc86fe0dd2f"} Jul 07 18:51:24.110670 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.110597 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" event={"ID":"f3010063-bc1a-4d37-95fb-c27e7d9720e7","Type":"ContainerStarted","Data":"b750992383e4995ae07f2eb4607936e05d3061dfb0963c81e5e2dcad26d37a29"} Jul 07 18:51:24.110670 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.110641 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" event={"ID":"f3010063-bc1a-4d37-95fb-c27e7d9720e7","Type":"ContainerStarted","Data":"98cd4c521027afb444e00bd391ec61809bd19ba9852a9e40da92d956f1c9c9e4"} Jul 07 18:51:24.112439 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.112405 2564 generic.go:356] "Generic (PLEG): container finished" podID="fe234b89-9fa1-4515-a421-e5f3e4393eae" containerID="2aa85128bf8496d0ebcfb4ee773bd6acc4fc8ca31be48e0a4f4201b53a954516" exitCode=0 Jul 07 18:51:24.112511 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.112475 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" event={"ID":"fe234b89-9fa1-4515-a421-e5f3e4393eae","Type":"ContainerDied","Data":"2aa85128bf8496d0ebcfb4ee773bd6acc4fc8ca31be48e0a4f4201b53a954516"} Jul 07 18:51:24.112571 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.112524 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" event={"ID":"fe234b89-9fa1-4515-a421-e5f3e4393eae","Type":"ContainerDied","Data":"45e6251028253c5134c235e247c12aa96fdb61dd5f3b595318c2415e132b5165"} Jul 07 18:51:24.112612 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.112575 2564 scope.go:117] "RemoveContainer" containerID="2aa85128bf8496d0ebcfb4ee773bd6acc4fc8ca31be48e0a4f4201b53a954516" Jul 07 18:51:24.112683 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.112650 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-569b766d4c-sdn6q" Jul 07 18:51:24.115070 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.115047 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe234b89-9fa1-4515-a421-e5f3e4393eae-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "fe234b89-9fa1-4515-a421-e5f3e4393eae" (UID: "fe234b89-9fa1-4515-a421-e5f3e4393eae"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:51:24.121623 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.121602 2564 scope.go:117] "RemoveContainer" containerID="2aa85128bf8496d0ebcfb4ee773bd6acc4fc8ca31be48e0a4f4201b53a954516" Jul 07 18:51:24.121877 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:51:24.121858 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2aa85128bf8496d0ebcfb4ee773bd6acc4fc8ca31be48e0a4f4201b53a954516\": container with ID starting with 2aa85128bf8496d0ebcfb4ee773bd6acc4fc8ca31be48e0a4f4201b53a954516 not found: ID does not exist" containerID="2aa85128bf8496d0ebcfb4ee773bd6acc4fc8ca31be48e0a4f4201b53a954516" Jul 07 18:51:24.121946 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.121883 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2aa85128bf8496d0ebcfb4ee773bd6acc4fc8ca31be48e0a4f4201b53a954516"} err="failed to get container status \"2aa85128bf8496d0ebcfb4ee773bd6acc4fc8ca31be48e0a4f4201b53a954516\": rpc error: code = NotFound desc = could not find container \"2aa85128bf8496d0ebcfb4ee773bd6acc4fc8ca31be48e0a4f4201b53a954516\": container with ID starting with 2aa85128bf8496d0ebcfb4ee773bd6acc4fc8ca31be48e0a4f4201b53a954516 not found: ID does not exist" Jul 07 18:51:24.179885 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.179851 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:51:24.185908 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.185873 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.189025 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.188998 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 18:51:24.189154 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.189028 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 18:51:24.189154 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.189005 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 18:51:24.189375 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.189359 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 18:51:24.189447 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.189427 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 18:51:24.189447 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.189440 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 18:51:24.189547 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.189447 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 18:51:24.189650 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.189634 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2nnonr638kt8q\"" Jul 07 18:51:24.189791 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.189771 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 18:51:24.189862 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.189835 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 18:51:24.189913 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.189873 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 18:51:24.189913 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.189887 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 18:51:24.190163 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.190131 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 18:51:24.190163 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.190144 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pzq75\"" Jul 07 18:51:24.190342 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.190173 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 18:51:24.191968 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.191951 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 18:51:24.195640 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.195622 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 18:51:24.202765 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.202745 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:51:24.204500 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.204473 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-console-serving-cert\") pod \"console-5d8759966d-pssgw\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.204924 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.204901 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-console-oauth-config\") pod \"console-5d8759966d-pssgw\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.204992 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.204947 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-service-ca\") pod \"console-5d8759966d-pssgw\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.205040 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.204990 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frs2n\" (UniqueName: \"kubernetes.io/projected/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-kube-api-access-frs2n\") pod \"console-5d8759966d-pssgw\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.205097 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.205043 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-console-config\") pod \"console-5d8759966d-pssgw\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.205097 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.205081 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-trusted-ca-bundle\") pod \"console-5d8759966d-pssgw\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.205266 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.205105 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-oauth-serving-cert\") pod \"console-5d8759966d-pssgw\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.205266 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.205159 2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-registry-tls\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:51:24.205266 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.205174 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2tmz\" (UniqueName: \"kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-kube-api-access-f2tmz\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:51:24.205266 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.205187 2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fe234b89-9fa1-4515-a421-e5f3e4393eae-installation-pull-secrets\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:51:24.205266 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.205215 2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fe234b89-9fa1-4515-a421-e5f3e4393eae-image-registry-private-configuration\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:51:24.205266 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.205228 2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fe234b89-9fa1-4515-a421-e5f3e4393eae-bound-sa-token\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:51:24.205266 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.205243 2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fe234b89-9fa1-4515-a421-e5f3e4393eae-ca-trust-extracted\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:51:24.205266 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.205254 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fe234b89-9fa1-4515-a421-e5f3e4393eae-trusted-ca\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:51:24.205884 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.205862 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-oauth-serving-cert\") pod \"console-5d8759966d-pssgw\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.206779 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.206755 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-console-config\") pod \"console-5d8759966d-pssgw\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.206880 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.206755 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-service-ca\") pod \"console-5d8759966d-pssgw\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.207396 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.207373 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-trusted-ca-bundle\") pod \"console-5d8759966d-pssgw\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.208985 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.208967 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-console-oauth-config\") pod \"console-5d8759966d-pssgw\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.209448 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.209429 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-console-serving-cert\") pod \"console-5d8759966d-pssgw\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.220913 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.220889 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frs2n\" (UniqueName: \"kubernetes.io/projected/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-kube-api-access-frs2n\") pod \"console-5d8759966d-pssgw\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.304333 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.304300 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:24.305561 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.305540 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.305610 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.305572 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.305610 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.305590 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-config\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.305679 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.305613 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.305710 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.305673 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.305740 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.305720 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c361ec7-0aef-4cf1-a191-28e4dc600802-config-out\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.305769 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.305743 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.305802 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.305782 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-web-config\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.305834 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.305806 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.305834 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.305822 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.305895 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.305847 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.305895 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.305865 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.305895 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.305885 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.306019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.305905 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.306019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.305964 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.306115 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.306043 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.306115 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.306093 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c361ec7-0aef-4cf1-a191-28e4dc600802-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.306225 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.306120 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.306225 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.306146 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.306225 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.306172 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwmss\" (UniqueName: \"kubernetes.io/projected/5c361ec7-0aef-4cf1-a191-28e4dc600802-kube-api-access-cwmss\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.406943 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.406912 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c361ec7-0aef-4cf1-a191-28e4dc600802-config-out\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.406943 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.406949 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.407167 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.406975 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-web-config\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.407167 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.407001 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.407987 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.407428 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.407987 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.407472 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.407987 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.407500 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.407987 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.407534 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.407987 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.407561 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.407987 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.407596 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.407987 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.407644 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.407987 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.407648 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.407987 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.407705 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c361ec7-0aef-4cf1-a191-28e4dc600802-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.407987 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.407734 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.407987 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.407762 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.407987 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.407787 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwmss\" (UniqueName: \"kubernetes.io/projected/5c361ec7-0aef-4cf1-a191-28e4dc600802-kube-api-access-cwmss\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.407987 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.407846 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.407987 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.407877 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.408782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.407908 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-config\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.408782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.407941 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.408782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.407980 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.408782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.408624 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.408782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.408747 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.411128 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.411051 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.411960 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.411518 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.411960 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.411832 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.412767 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.412084 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.414846 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.413483 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.414846 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.413579 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.416174 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.415759 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c361ec7-0aef-4cf1-a191-28e4dc600802-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.416174 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.415799 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.416174 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.416143 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-config\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.416424 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.416400 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.416898 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.416502 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.417377 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.417288 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.417457 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.417387 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-web-config\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.417692 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.417669 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c361ec7-0aef-4cf1-a191-28e4dc600802-config-out\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.418347 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.418327 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.419647 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.419620 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.426323 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.426133 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwmss\" (UniqueName: \"kubernetes.io/projected/5c361ec7-0aef-4cf1-a191-28e4dc600802-kube-api-access-cwmss\") pod \"prometheus-k8s-0\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.443421 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.443387 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-569b766d4c-sdn6q"] Jul 07 18:51:24.446454 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.446427 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-569b766d4c-sdn6q"] Jul 07 18:51:24.452249 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.452231 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d8759966d-pssgw"] Jul 07 18:51:24.455482 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:51:24.455455 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd720eaf5_7915_48b7_8f9b_8b3e7eaaec46.slice/crio-79187be52680caeba71c6a7c113e94cabbe0f92b5b419f9137cda537c921732d WatchSource:0}: Error finding container 79187be52680caeba71c6a7c113e94cabbe0f92b5b419f9137cda537c921732d: Status 404 returned error can't find the container with id 79187be52680caeba71c6a7c113e94cabbe0f92b5b419f9137cda537c921732d Jul 07 18:51:24.498285 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.498261 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:24.575391 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.575362 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe234b89-9fa1-4515-a421-e5f3e4393eae" path="/var/lib/kubelet/pods/fe234b89-9fa1-4515-a421-e5f3e4393eae/volumes" Jul 07 18:51:24.662123 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:24.662048 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:51:24.666071 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:51:24.666039 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c361ec7_0aef_4cf1_a191_28e4dc600802.slice/crio-c3360a15e8d450b70837c8c4031642b372bb30e3717a4390a9329189fc222bbc WatchSource:0}: Error finding container c3360a15e8d450b70837c8c4031642b372bb30e3717a4390a9329189fc222bbc: Status 404 returned error can't find the container with id c3360a15e8d450b70837c8c4031642b372bb30e3717a4390a9329189fc222bbc Jul 07 18:51:25.121164 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:25.121132 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8759966d-pssgw" event={"ID":"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46","Type":"ContainerStarted","Data":"05c509770cff1d47c0f62543ea744acac7d5def504480a246048d79adbf03adc"} Jul 07 18:51:25.121283 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:25.121188 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8759966d-pssgw" event={"ID":"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46","Type":"ContainerStarted","Data":"79187be52680caeba71c6a7c113e94cabbe0f92b5b419f9137cda537c921732d"} Jul 07 18:51:25.122613 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:25.122551 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c361ec7-0aef-4cf1-a191-28e4dc600802","Type":"ContainerStarted","Data":"c3360a15e8d450b70837c8c4031642b372bb30e3717a4390a9329189fc222bbc"} Jul 07 18:51:25.126612 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:25.126588 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" event={"ID":"f3010063-bc1a-4d37-95fb-c27e7d9720e7","Type":"ContainerStarted","Data":"2589f909c7c3d0745b6e633cb283efddc88475991036b44b283e924680107560"} Jul 07 18:51:25.126703 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:25.126657 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" event={"ID":"f3010063-bc1a-4d37-95fb-c27e7d9720e7","Type":"ContainerStarted","Data":"458389f040df6c696d04bf1bed30935ae071bdfd8cb1ff0295de05704710b3c5"} Jul 07 18:51:25.144583 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:25.144520 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d8759966d-pssgw" podStartSLOduration=2.144499637 podStartE2EDuration="2.144499637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:51:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:25.142766878 +0000 UTC m=+97.112402895" watchObservedRunningTime="2026-07-07 18:51:25.144499637 +0000 UTC m=+97.114135655" Jul 07 18:51:26.132274 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:26.132242 2564 generic.go:356] "Generic (PLEG): container finished" podID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerID="7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890" exitCode=0 Jul 07 18:51:26.132708 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:26.132338 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c361ec7-0aef-4cf1-a191-28e4dc600802","Type":"ContainerDied","Data":"7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890"} Jul 07 18:51:26.135503 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:26.135479 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" event={"ID":"f3010063-bc1a-4d37-95fb-c27e7d9720e7","Type":"ContainerStarted","Data":"fe4b76d3b2641b05f622aa3742bc0bae38a18635b7faa4f598d149e8cb8344e9"} Jul 07 18:51:26.188542 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:26.188492 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" podStartSLOduration=2.852557206 podStartE2EDuration="6.188476435s" totalImagesPullingTime="3.335919229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:51:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:26.187827736 +0000 UTC m=+98.157463753" watchObservedRunningTime="2026-07-07 18:51:26.188476435 +0000 UTC m=+98.158112449" Jul 07 18:51:27.140493 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:27.140461 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:31.159510 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:31.159438 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c361ec7-0aef-4cf1-a191-28e4dc600802","Type":"ContainerStarted","Data":"7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d"} Jul 07 18:51:31.159510 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:31.159497 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c361ec7-0aef-4cf1-a191-28e4dc600802","Type":"ContainerStarted","Data":"d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f"} Jul 07 18:51:31.159905 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:31.159526 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c361ec7-0aef-4cf1-a191-28e4dc600802","Type":"ContainerStarted","Data":"16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f"} Jul 07 18:51:31.159905 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:31.159550 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c361ec7-0aef-4cf1-a191-28e4dc600802","Type":"ContainerStarted","Data":"d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4"} Jul 07 18:51:31.159905 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:31.159573 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c361ec7-0aef-4cf1-a191-28e4dc600802","Type":"ContainerStarted","Data":"a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee"} Jul 07 18:51:31.159905 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:31.159594 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c361ec7-0aef-4cf1-a191-28e4dc600802","Type":"ContainerStarted","Data":"b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39"} Jul 07 18:51:31.200182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:31.200125 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.219293987 podStartE2EDuration="7.200108679s" totalImagesPullingTime="4.961800722s" totalInitContainerRuntime="19.01397ms" isStatefulPod=false podCreationTimestamp="2026-07-07 18:51:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:31.197596597 +0000 UTC m=+103.167232625" watchObservedRunningTime="2026-07-07 18:51:31.200108679 +0000 UTC m=+103.169744696" Jul 07 18:51:32.822699 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:32.822660 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d8759966d-pssgw"] Jul 07 18:51:32.866345 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:32.866315 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f8d9549fc-b88tb"] Jul 07 18:51:32.910292 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:32.910256 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f8d9549fc-b88tb"] Jul 07 18:51:32.910467 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:32.910370 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:32.975001 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:32.974962 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca83df20-4431-4e6c-bf97-00861641ad97-console-oauth-config\") pod \"console-5f8d9549fc-b88tb\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:32.975001 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:32.974998 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca83df20-4431-4e6c-bf97-00861641ad97-console-serving-cert\") pod \"console-5f8d9549fc-b88tb\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:32.975320 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:32.975021 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-trusted-ca-bundle\") pod \"console-5f8d9549fc-b88tb\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:32.975320 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:32.975080 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-console-config\") pod \"console-5f8d9549fc-b88tb\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:32.975320 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:32.975096 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-service-ca\") pod \"console-5f8d9549fc-b88tb\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:32.975320 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:32.975168 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-oauth-serving-cert\") pod \"console-5f8d9549fc-b88tb\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:32.975320 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:32.975191 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-px5vn\" (UniqueName: \"kubernetes.io/projected/ca83df20-4431-4e6c-bf97-00861641ad97-kube-api-access-px5vn\") pod \"console-5f8d9549fc-b88tb\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:33.076140 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:33.076065 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-console-config\") pod \"console-5f8d9549fc-b88tb\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:33.076140 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:33.076098 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-service-ca\") pod \"console-5f8d9549fc-b88tb\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:33.076140 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:33.076130 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-oauth-serving-cert\") pod \"console-5f8d9549fc-b88tb\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:33.076408 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:33.076154 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-px5vn\" (UniqueName: \"kubernetes.io/projected/ca83df20-4431-4e6c-bf97-00861641ad97-kube-api-access-px5vn\") pod \"console-5f8d9549fc-b88tb\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:33.076408 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:33.076224 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca83df20-4431-4e6c-bf97-00861641ad97-console-oauth-config\") pod \"console-5f8d9549fc-b88tb\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:33.076408 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:33.076249 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca83df20-4431-4e6c-bf97-00861641ad97-console-serving-cert\") pod \"console-5f8d9549fc-b88tb\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:33.076408 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:33.076275 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-trusted-ca-bundle\") pod \"console-5f8d9549fc-b88tb\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:33.076955 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:33.076923 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-service-ca\") pod \"console-5f8d9549fc-b88tb\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:33.077092 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:33.076977 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-console-config\") pod \"console-5f8d9549fc-b88tb\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:33.077092 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:33.076975 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-oauth-serving-cert\") pod \"console-5f8d9549fc-b88tb\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:33.077225 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:33.077131 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-trusted-ca-bundle\") pod \"console-5f8d9549fc-b88tb\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:33.078791 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:33.078770 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca83df20-4431-4e6c-bf97-00861641ad97-console-oauth-config\") pod \"console-5f8d9549fc-b88tb\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:33.078997 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:33.078974 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca83df20-4431-4e6c-bf97-00861641ad97-console-serving-cert\") pod \"console-5f8d9549fc-b88tb\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:33.092175 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:33.092151 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-px5vn\" (UniqueName: \"kubernetes.io/projected/ca83df20-4431-4e6c-bf97-00861641ad97-kube-api-access-px5vn\") pod \"console-5f8d9549fc-b88tb\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:33.150124 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:33.150104 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-8488ddff78-wthfd" Jul 07 18:51:33.221625 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:33.221589 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:33.405516 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:33.405490 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f8d9549fc-b88tb"] Jul 07 18:51:33.408382 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:51:33.408346 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca83df20_4431_4e6c_bf97_00861641ad97.slice/crio-0232a3cdb394d3a13cc23ce97b3c3f182fde0218c1d4113d6bc17793183bbe5f WatchSource:0}: Error finding container 0232a3cdb394d3a13cc23ce97b3c3f182fde0218c1d4113d6bc17793183bbe5f: Status 404 returned error can't find the container with id 0232a3cdb394d3a13cc23ce97b3c3f182fde0218c1d4113d6bc17793183bbe5f Jul 07 18:51:33.997589 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:33.997559 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mkg6r" Jul 07 18:51:34.174811 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:34.174775 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f8d9549fc-b88tb" event={"ID":"ca83df20-4431-4e6c-bf97-00861641ad97","Type":"ContainerStarted","Data":"dff112c05d2ef4e143cac927f1c4bfeb653abbea279d2b94405a86d869d86870"} Jul 07 18:51:34.174965 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:34.174826 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f8d9549fc-b88tb" event={"ID":"ca83df20-4431-4e6c-bf97-00861641ad97","Type":"ContainerStarted","Data":"0232a3cdb394d3a13cc23ce97b3c3f182fde0218c1d4113d6bc17793183bbe5f"} Jul 07 18:51:34.197494 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:34.197440 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f8d9549fc-b88tb" podStartSLOduration=2.197420868 podStartE2EDuration="2.197420868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:51:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:51:34.195321817 +0000 UTC m=+106.164957855" watchObservedRunningTime="2026-07-07 18:51:34.197420868 +0000 UTC m=+106.167056890" Jul 07 18:51:34.304745 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:34.304657 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:34.498653 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:34.498622 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:51:43.222511 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:43.222475 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:43.222972 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:43.222543 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:43.227185 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:43.227164 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:44.224211 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:44.224170 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:51:44.285423 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:44.285390 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b54b55556-kmbdb"] Jul 07 18:51:45.226093 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:45.226060 2564 generic.go:356] "Generic (PLEG): container finished" podID="1ccc5d0f-1bc6-4866-b99a-82252b234d0f" containerID="f105ef8f12c235f57694629feadbe50efae55990a60b58b08d490d532741f79a" exitCode=0 Jul 07 18:51:45.226453 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:45.226130 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" event={"ID":"1ccc5d0f-1bc6-4866-b99a-82252b234d0f","Type":"ContainerDied","Data":"f105ef8f12c235f57694629feadbe50efae55990a60b58b08d490d532741f79a"} Jul 07 18:51:45.226649 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:45.226632 2564 scope.go:117] "RemoveContainer" containerID="f105ef8f12c235f57694629feadbe50efae55990a60b58b08d490d532741f79a" Jul 07 18:51:46.232101 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:46.232073 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-trfsb" event={"ID":"1ccc5d0f-1bc6-4866-b99a-82252b234d0f","Type":"ContainerStarted","Data":"eb999245a5d28bab561ee72810d929b4d0a989a112639c3b1b41a851fbf78451"} Jul 07 18:51:57.843599 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:57.843558 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5d8759966d-pssgw" podUID="d720eaf5-7915-48b7-8f9b-8b3e7eaaec46" containerName="console" containerID="cri-o://05c509770cff1d47c0f62543ea744acac7d5def504480a246048d79adbf03adc" gracePeriod=15 Jul 07 18:51:58.127816 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.127789 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:58.181626 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.181590 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-service-ca\") pod \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " Jul 07 18:51:58.181811 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.181658 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frs2n\" (UniqueName: \"kubernetes.io/projected/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-kube-api-access-frs2n\") pod \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " Jul 07 18:51:58.181811 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.181704 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-console-serving-cert\") pod \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " Jul 07 18:51:58.181811 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.181735 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-console-config\") pod \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " Jul 07 18:51:58.181811 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.181764 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-trusted-ca-bundle\") pod \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " Jul 07 18:51:58.182036 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.181825 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-oauth-serving-cert\") pod \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " Jul 07 18:51:58.182036 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.181851 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-console-oauth-config\") pod \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\" (UID: \"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46\") " Jul 07 18:51:58.182036 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.181956 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-service-ca" (OuterVolumeSpecName: "service-ca") pod "d720eaf5-7915-48b7-8f9b-8b3e7eaaec46" (UID: "d720eaf5-7915-48b7-8f9b-8b3e7eaaec46"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:51:58.182036 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.182028 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-service-ca\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:51:58.182284 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.182262 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-console-config" (OuterVolumeSpecName: "console-config") pod "d720eaf5-7915-48b7-8f9b-8b3e7eaaec46" (UID: "d720eaf5-7915-48b7-8f9b-8b3e7eaaec46"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:51:58.182356 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.182336 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d720eaf5-7915-48b7-8f9b-8b3e7eaaec46" (UID: "d720eaf5-7915-48b7-8f9b-8b3e7eaaec46"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:51:58.182513 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.182487 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d720eaf5-7915-48b7-8f9b-8b3e7eaaec46" (UID: "d720eaf5-7915-48b7-8f9b-8b3e7eaaec46"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:51:58.184497 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.184461 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d720eaf5-7915-48b7-8f9b-8b3e7eaaec46" (UID: "d720eaf5-7915-48b7-8f9b-8b3e7eaaec46"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:51:58.184497 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.184481 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-kube-api-access-frs2n" (OuterVolumeSpecName: "kube-api-access-frs2n") pod "d720eaf5-7915-48b7-8f9b-8b3e7eaaec46" (UID: "d720eaf5-7915-48b7-8f9b-8b3e7eaaec46"). InnerVolumeSpecName "kube-api-access-frs2n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:51:58.184845 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.184820 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d720eaf5-7915-48b7-8f9b-8b3e7eaaec46" (UID: "d720eaf5-7915-48b7-8f9b-8b3e7eaaec46"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:51:58.283728 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.283687 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-oauth-serving-cert\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:51:58.283728 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.283722 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-console-oauth-config\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:51:58.283982 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.283736 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frs2n\" (UniqueName: \"kubernetes.io/projected/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-kube-api-access-frs2n\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:51:58.283982 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.283750 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-console-serving-cert\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:51:58.283982 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.283763 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-console-config\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:51:58.283982 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.283775 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46-trusted-ca-bundle\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:51:58.286646 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.286620 2564 generic.go:356] "Generic (PLEG): container finished" podID="d720eaf5-7915-48b7-8f9b-8b3e7eaaec46" containerID="05c509770cff1d47c0f62543ea744acac7d5def504480a246048d79adbf03adc" exitCode=0 Jul 07 18:51:58.286754 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.286692 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8759966d-pssgw" Jul 07 18:51:58.286754 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.286713 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8759966d-pssgw" event={"ID":"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46","Type":"ContainerDied","Data":"05c509770cff1d47c0f62543ea744acac7d5def504480a246048d79adbf03adc"} Jul 07 18:51:58.286835 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.286796 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8759966d-pssgw" event={"ID":"d720eaf5-7915-48b7-8f9b-8b3e7eaaec46","Type":"ContainerDied","Data":"79187be52680caeba71c6a7c113e94cabbe0f92b5b419f9137cda537c921732d"} Jul 07 18:51:58.286835 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.286823 2564 scope.go:117] "RemoveContainer" containerID="05c509770cff1d47c0f62543ea744acac7d5def504480a246048d79adbf03adc" Jul 07 18:51:58.309389 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.308456 2564 scope.go:117] "RemoveContainer" containerID="05c509770cff1d47c0f62543ea744acac7d5def504480a246048d79adbf03adc" Jul 07 18:51:58.309638 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:51:58.309609 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05c509770cff1d47c0f62543ea744acac7d5def504480a246048d79adbf03adc\": container with ID starting with 05c509770cff1d47c0f62543ea744acac7d5def504480a246048d79adbf03adc not found: ID does not exist" containerID="05c509770cff1d47c0f62543ea744acac7d5def504480a246048d79adbf03adc" Jul 07 18:51:58.309710 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.309649 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05c509770cff1d47c0f62543ea744acac7d5def504480a246048d79adbf03adc"} err="failed to get container status \"05c509770cff1d47c0f62543ea744acac7d5def504480a246048d79adbf03adc\": rpc error: code = NotFound desc = could not find container \"05c509770cff1d47c0f62543ea744acac7d5def504480a246048d79adbf03adc\": container with ID starting with 05c509770cff1d47c0f62543ea744acac7d5def504480a246048d79adbf03adc not found: ID does not exist" Jul 07 18:51:58.322668 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.322642 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d8759966d-pssgw"] Jul 07 18:51:58.329152 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.329128 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d8759966d-pssgw"] Jul 07 18:51:58.572591 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:51:58.572553 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d720eaf5-7915-48b7-8f9b-8b3e7eaaec46" path="/var/lib/kubelet/pods/d720eaf5-7915-48b7-8f9b-8b3e7eaaec46/volumes" Jul 07 18:52:06.321429 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:06.321340 2564 generic.go:356] "Generic (PLEG): container finished" podID="7049298b-743d-4673-a84f-4e2465166aa3" containerID="f06c1d31a57cf0a273182e0f807670bf855ca1331892a2759f5600a4428cc71d" exitCode=0 Jul 07 18:52:06.321429 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:06.321409 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d4mz6" event={"ID":"7049298b-743d-4673-a84f-4e2465166aa3","Type":"ContainerDied","Data":"f06c1d31a57cf0a273182e0f807670bf855ca1331892a2759f5600a4428cc71d"} Jul 07 18:52:06.321818 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:06.321705 2564 scope.go:117] "RemoveContainer" containerID="f06c1d31a57cf0a273182e0f807670bf855ca1331892a2759f5600a4428cc71d" Jul 07 18:52:07.326899 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:07.326864 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-d4mz6" event={"ID":"7049298b-743d-4673-a84f-4e2465166aa3","Type":"ContainerStarted","Data":"4e1be2d80a4d6f22151c6c27da18eb7989d5ac497840f7db1769e14328ce83a9"} Jul 07 18:52:09.305015 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:09.304979 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5b54b55556-kmbdb" podUID="fe8ffd85-f400-45e4-b88d-457e745d9132" containerName="console" containerID="cri-o://8c391c48cbbc88161399a1e278167870c0ddc653c98c098638e6230f6068e737" gracePeriod=15 Jul 07 18:52:09.558572 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:09.558520 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:52:09.680690 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:09.680663 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe8ffd85-f400-45e4-b88d-457e745d9132-console-config\") pod \"fe8ffd85-f400-45e4-b88d-457e745d9132\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " Jul 07 18:52:09.680826 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:09.680733 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe8ffd85-f400-45e4-b88d-457e745d9132-oauth-serving-cert\") pod \"fe8ffd85-f400-45e4-b88d-457e745d9132\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " Jul 07 18:52:09.680826 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:09.680792 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe8ffd85-f400-45e4-b88d-457e745d9132-service-ca\") pod \"fe8ffd85-f400-45e4-b88d-457e745d9132\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " Jul 07 18:52:09.680826 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:09.680817 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmrg4\" (UniqueName: \"kubernetes.io/projected/fe8ffd85-f400-45e4-b88d-457e745d9132-kube-api-access-dmrg4\") pod \"fe8ffd85-f400-45e4-b88d-457e745d9132\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " Jul 07 18:52:09.680943 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:09.680861 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe8ffd85-f400-45e4-b88d-457e745d9132-console-serving-cert\") pod \"fe8ffd85-f400-45e4-b88d-457e745d9132\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " Jul 07 18:52:09.680943 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:09.680898 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe8ffd85-f400-45e4-b88d-457e745d9132-console-oauth-config\") pod \"fe8ffd85-f400-45e4-b88d-457e745d9132\" (UID: \"fe8ffd85-f400-45e4-b88d-457e745d9132\") " Jul 07 18:52:09.681135 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:09.681103 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe8ffd85-f400-45e4-b88d-457e745d9132-console-config" (OuterVolumeSpecName: "console-config") pod "fe8ffd85-f400-45e4-b88d-457e745d9132" (UID: "fe8ffd85-f400-45e4-b88d-457e745d9132"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:52:09.681272 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:09.681174 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe8ffd85-f400-45e4-b88d-457e745d9132-service-ca" (OuterVolumeSpecName: "service-ca") pod "fe8ffd85-f400-45e4-b88d-457e745d9132" (UID: "fe8ffd85-f400-45e4-b88d-457e745d9132"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:52:09.681272 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:09.681223 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe8ffd85-f400-45e4-b88d-457e745d9132-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fe8ffd85-f400-45e4-b88d-457e745d9132" (UID: "fe8ffd85-f400-45e4-b88d-457e745d9132"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:52:09.683047 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:09.683020 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe8ffd85-f400-45e4-b88d-457e745d9132-kube-api-access-dmrg4" (OuterVolumeSpecName: "kube-api-access-dmrg4") pod "fe8ffd85-f400-45e4-b88d-457e745d9132" (UID: "fe8ffd85-f400-45e4-b88d-457e745d9132"). InnerVolumeSpecName "kube-api-access-dmrg4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:52:09.683047 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:09.683030 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe8ffd85-f400-45e4-b88d-457e745d9132-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fe8ffd85-f400-45e4-b88d-457e745d9132" (UID: "fe8ffd85-f400-45e4-b88d-457e745d9132"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:52:09.683179 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:09.683042 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe8ffd85-f400-45e4-b88d-457e745d9132-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fe8ffd85-f400-45e4-b88d-457e745d9132" (UID: "fe8ffd85-f400-45e4-b88d-457e745d9132"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:52:09.781836 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:09.781816 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fe8ffd85-f400-45e4-b88d-457e745d9132-console-config\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:09.781836 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:09.781836 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fe8ffd85-f400-45e4-b88d-457e745d9132-oauth-serving-cert\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:09.781958 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:09.781846 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fe8ffd85-f400-45e4-b88d-457e745d9132-service-ca\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:09.781958 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:09.781855 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmrg4\" (UniqueName: \"kubernetes.io/projected/fe8ffd85-f400-45e4-b88d-457e745d9132-kube-api-access-dmrg4\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:09.781958 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:09.781864 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fe8ffd85-f400-45e4-b88d-457e745d9132-console-serving-cert\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:09.781958 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:09.781873 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fe8ffd85-f400-45e4-b88d-457e745d9132-console-oauth-config\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:10.339314 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:10.339276 2564 generic.go:356] "Generic (PLEG): container finished" podID="fe8ffd85-f400-45e4-b88d-457e745d9132" containerID="8c391c48cbbc88161399a1e278167870c0ddc653c98c098638e6230f6068e737" exitCode=0 Jul 07 18:52:10.339696 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:10.339322 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b54b55556-kmbdb" Jul 07 18:52:10.339696 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:10.339354 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b54b55556-kmbdb" event={"ID":"fe8ffd85-f400-45e4-b88d-457e745d9132","Type":"ContainerDied","Data":"8c391c48cbbc88161399a1e278167870c0ddc653c98c098638e6230f6068e737"} Jul 07 18:52:10.339696 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:10.339405 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b54b55556-kmbdb" event={"ID":"fe8ffd85-f400-45e4-b88d-457e745d9132","Type":"ContainerDied","Data":"80e387ae880ed0018789eb8639df6a96d5bb385d2d55d9443fbccc1424840a62"} Jul 07 18:52:10.339696 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:10.339427 2564 scope.go:117] "RemoveContainer" containerID="8c391c48cbbc88161399a1e278167870c0ddc653c98c098638e6230f6068e737" Jul 07 18:52:10.348472 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:10.348455 2564 scope.go:117] "RemoveContainer" containerID="8c391c48cbbc88161399a1e278167870c0ddc653c98c098638e6230f6068e737" Jul 07 18:52:10.348706 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:52:10.348688 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c391c48cbbc88161399a1e278167870c0ddc653c98c098638e6230f6068e737\": container with ID starting with 8c391c48cbbc88161399a1e278167870c0ddc653c98c098638e6230f6068e737 not found: ID does not exist" containerID="8c391c48cbbc88161399a1e278167870c0ddc653c98c098638e6230f6068e737" Jul 07 18:52:10.348752 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:10.348715 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c391c48cbbc88161399a1e278167870c0ddc653c98c098638e6230f6068e737"} err="failed to get container status \"8c391c48cbbc88161399a1e278167870c0ddc653c98c098638e6230f6068e737\": rpc error: code = NotFound desc = could not find container \"8c391c48cbbc88161399a1e278167870c0ddc653c98c098638e6230f6068e737\": container with ID starting with 8c391c48cbbc88161399a1e278167870c0ddc653c98c098638e6230f6068e737 not found: ID does not exist" Jul 07 18:52:10.368305 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:10.368282 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b54b55556-kmbdb"] Jul 07 18:52:10.373219 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:10.373185 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b54b55556-kmbdb"] Jul 07 18:52:10.572793 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:10.572755 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe8ffd85-f400-45e4-b88d-457e745d9132" path="/var/lib/kubelet/pods/fe8ffd85-f400-45e4-b88d-457e745d9132/volumes" Jul 07 18:52:24.499440 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:24.499411 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:24.517425 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:24.517402 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:25.411580 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:25.411554 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:42.654846 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.654754 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:52:42.655422 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.655038 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="prometheus" containerID="cri-o://b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39" gracePeriod=600 Jul 07 18:52:42.655422 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.655053 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="kube-rbac-proxy" containerID="cri-o://d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f" gracePeriod=600 Jul 07 18:52:42.655422 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.655106 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="config-reloader" containerID="cri-o://a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee" gracePeriod=600 Jul 07 18:52:42.655422 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.655140 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="kube-rbac-proxy-web" containerID="cri-o://16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f" gracePeriod=600 Jul 07 18:52:42.655422 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.655154 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="kube-rbac-proxy-thanos" containerID="cri-o://7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d" gracePeriod=600 Jul 07 18:52:42.655422 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.655090 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="thanos-sidecar" containerID="cri-o://d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4" gracePeriod=600 Jul 07 18:52:42.911302 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.910545 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:42.929210 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.929161 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-web-config\") pod \"5c361ec7-0aef-4cf1-a191-28e4dc600802\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " Jul 07 18:52:42.929369 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.929217 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"5c361ec7-0aef-4cf1-a191-28e4dc600802\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " Jul 07 18:52:42.929369 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.929238 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-thanos-prometheus-http-client-file\") pod \"5c361ec7-0aef-4cf1-a191-28e4dc600802\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " Jul 07 18:52:42.929369 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.929256 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwmss\" (UniqueName: \"kubernetes.io/projected/5c361ec7-0aef-4cf1-a191-28e4dc600802-kube-api-access-cwmss\") pod \"5c361ec7-0aef-4cf1-a191-28e4dc600802\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " Jul 07 18:52:42.929369 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.929275 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-kube-rbac-proxy\") pod \"5c361ec7-0aef-4cf1-a191-28e4dc600802\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " Jul 07 18:52:42.929369 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.929313 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-trusted-ca-bundle\") pod \"5c361ec7-0aef-4cf1-a191-28e4dc600802\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " Jul 07 18:52:42.929369 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.929348 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-config\") pod \"5c361ec7-0aef-4cf1-a191-28e4dc600802\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " Jul 07 18:52:42.930077 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.929710 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-db\") pod \"5c361ec7-0aef-4cf1-a191-28e4dc600802\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " Jul 07 18:52:42.930077 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.929757 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-rulefiles-2\") pod \"5c361ec7-0aef-4cf1-a191-28e4dc600802\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " Jul 07 18:52:42.930077 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.929789 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-configmap-metrics-client-ca\") pod \"5c361ec7-0aef-4cf1-a191-28e4dc600802\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " Jul 07 18:52:42.930077 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.929815 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-grpc-tls\") pod \"5c361ec7-0aef-4cf1-a191-28e4dc600802\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " Jul 07 18:52:42.930077 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.929810 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "5c361ec7-0aef-4cf1-a191-28e4dc600802" (UID: "5c361ec7-0aef-4cf1-a191-28e4dc600802"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:52:42.930077 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.929840 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-configmap-serving-certs-ca-bundle\") pod \"5c361ec7-0aef-4cf1-a191-28e4dc600802\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " Jul 07 18:52:42.930077 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.930054 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "5c361ec7-0aef-4cf1-a191-28e4dc600802" (UID: "5c361ec7-0aef-4cf1-a191-28e4dc600802"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:52:42.930510 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.930099 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-rulefiles-1\") pod \"5c361ec7-0aef-4cf1-a191-28e4dc600802\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " Jul 07 18:52:42.930510 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.930151 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"5c361ec7-0aef-4cf1-a191-28e4dc600802\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " Jul 07 18:52:42.930510 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.930183 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c361ec7-0aef-4cf1-a191-28e4dc600802-tls-assets\") pod \"5c361ec7-0aef-4cf1-a191-28e4dc600802\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " Jul 07 18:52:42.930510 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.930281 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-configmap-kubelet-serving-ca-bundle\") pod \"5c361ec7-0aef-4cf1-a191-28e4dc600802\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " Jul 07 18:52:42.930510 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.930308 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-rulefiles-0\") pod \"5c361ec7-0aef-4cf1-a191-28e4dc600802\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " Jul 07 18:52:42.930510 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.930341 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c361ec7-0aef-4cf1-a191-28e4dc600802-config-out\") pod \"5c361ec7-0aef-4cf1-a191-28e4dc600802\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " Jul 07 18:52:42.930510 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.930375 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-metrics-client-certs\") pod \"5c361ec7-0aef-4cf1-a191-28e4dc600802\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " Jul 07 18:52:42.930510 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.930402 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-prometheus-k8s-tls\") pod \"5c361ec7-0aef-4cf1-a191-28e4dc600802\" (UID: \"5c361ec7-0aef-4cf1-a191-28e4dc600802\") " Jul 07 18:52:42.930910 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.930655 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-trusted-ca-bundle\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:42.930910 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.930674 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:42.932921 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.930147 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "5c361ec7-0aef-4cf1-a191-28e4dc600802" (UID: "5c361ec7-0aef-4cf1-a191-28e4dc600802"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:52:42.932921 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.930400 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "5c361ec7-0aef-4cf1-a191-28e4dc600802" (UID: "5c361ec7-0aef-4cf1-a191-28e4dc600802"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:52:42.932921 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.931759 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "5c361ec7-0aef-4cf1-a191-28e4dc600802" (UID: "5c361ec7-0aef-4cf1-a191-28e4dc600802"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:52:42.932921 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.932134 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "5c361ec7-0aef-4cf1-a191-28e4dc600802" (UID: "5c361ec7-0aef-4cf1-a191-28e4dc600802"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:52:42.932921 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.932209 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-config" (OuterVolumeSpecName: "config") pod "5c361ec7-0aef-4cf1-a191-28e4dc600802" (UID: "5c361ec7-0aef-4cf1-a191-28e4dc600802"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:52:42.932921 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.932402 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "5c361ec7-0aef-4cf1-a191-28e4dc600802" (UID: "5c361ec7-0aef-4cf1-a191-28e4dc600802"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:52:42.932921 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.932750 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "5c361ec7-0aef-4cf1-a191-28e4dc600802" (UID: "5c361ec7-0aef-4cf1-a191-28e4dc600802"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:52:42.932921 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.932795 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "5c361ec7-0aef-4cf1-a191-28e4dc600802" (UID: "5c361ec7-0aef-4cf1-a191-28e4dc600802"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:52:42.933823 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.933782 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "5c361ec7-0aef-4cf1-a191-28e4dc600802" (UID: "5c361ec7-0aef-4cf1-a191-28e4dc600802"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:52:42.933925 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.933886 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "5c361ec7-0aef-4cf1-a191-28e4dc600802" (UID: "5c361ec7-0aef-4cf1-a191-28e4dc600802"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:52:42.934126 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.934102 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "5c361ec7-0aef-4cf1-a191-28e4dc600802" (UID: "5c361ec7-0aef-4cf1-a191-28e4dc600802"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:52:42.934215 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.934123 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "5c361ec7-0aef-4cf1-a191-28e4dc600802" (UID: "5c361ec7-0aef-4cf1-a191-28e4dc600802"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:52:42.934437 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.934411 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c361ec7-0aef-4cf1-a191-28e4dc600802-kube-api-access-cwmss" (OuterVolumeSpecName: "kube-api-access-cwmss") pod "5c361ec7-0aef-4cf1-a191-28e4dc600802" (UID: "5c361ec7-0aef-4cf1-a191-28e4dc600802"). InnerVolumeSpecName "kube-api-access-cwmss". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:52:42.934516 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.934415 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c361ec7-0aef-4cf1-a191-28e4dc600802-config-out" (OuterVolumeSpecName: "config-out") pod "5c361ec7-0aef-4cf1-a191-28e4dc600802" (UID: "5c361ec7-0aef-4cf1-a191-28e4dc600802"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:52:42.935655 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.935628 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "5c361ec7-0aef-4cf1-a191-28e4dc600802" (UID: "5c361ec7-0aef-4cf1-a191-28e4dc600802"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:52:42.936028 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.936000 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c361ec7-0aef-4cf1-a191-28e4dc600802-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5c361ec7-0aef-4cf1-a191-28e4dc600802" (UID: "5c361ec7-0aef-4cf1-a191-28e4dc600802"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:52:42.936126 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.936052 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "5c361ec7-0aef-4cf1-a191-28e4dc600802" (UID: "5c361ec7-0aef-4cf1-a191-28e4dc600802"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:52:42.945095 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:42.945066 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-web-config" (OuterVolumeSpecName: "web-config") pod "5c361ec7-0aef-4cf1-a191-28e4dc600802" (UID: "5c361ec7-0aef-4cf1-a191-28e4dc600802"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:52:43.031347 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.031316 2564 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:43.031347 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.031341 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:43.031347 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.031351 2564 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5c361ec7-0aef-4cf1-a191-28e4dc600802-config-out\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:43.031588 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.031360 2564 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-metrics-client-certs\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:43.031588 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.031370 2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-prometheus-k8s-tls\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:43.031588 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.031381 2564 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-web-config\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:43.031588 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.031389 2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:43.031588 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.031397 2564 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-thanos-prometheus-http-client-file\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:43.031588 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.031407 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwmss\" (UniqueName: \"kubernetes.io/projected/5c361ec7-0aef-4cf1-a191-28e4dc600802-kube-api-access-cwmss\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:43.031588 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.031416 2564 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-kube-rbac-proxy\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:43.031588 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.031424 2564 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-config\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:43.031588 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.031432 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-db\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:43.031588 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.031440 2564 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-configmap-metrics-client-ca\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:43.031588 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.031449 2564 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-grpc-tls\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:43.031588 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.031459 2564 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:43.031588 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.031466 2564 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5c361ec7-0aef-4cf1-a191-28e4dc600802-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:43.031588 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.031476 2564 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5c361ec7-0aef-4cf1-a191-28e4dc600802-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:43.031588 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.031485 2564 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5c361ec7-0aef-4cf1-a191-28e4dc600802-tls-assets\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:52:43.472371 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.472337 2564 generic.go:356] "Generic (PLEG): container finished" podID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerID="7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d" exitCode=0 Jul 07 18:52:43.472371 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.472362 2564 generic.go:356] "Generic (PLEG): container finished" podID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerID="d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f" exitCode=0 Jul 07 18:52:43.472371 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.472368 2564 generic.go:356] "Generic (PLEG): container finished" podID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerID="16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f" exitCode=0 Jul 07 18:52:43.472371 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.472374 2564 generic.go:356] "Generic (PLEG): container finished" podID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerID="d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4" exitCode=0 Jul 07 18:52:43.472371 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.472379 2564 generic.go:356] "Generic (PLEG): container finished" podID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerID="a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee" exitCode=0 Jul 07 18:52:43.472777 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.472385 2564 generic.go:356] "Generic (PLEG): container finished" podID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerID="b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39" exitCode=0 Jul 07 18:52:43.472777 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.472417 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.472777 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.472417 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c361ec7-0aef-4cf1-a191-28e4dc600802","Type":"ContainerDied","Data":"7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d"} Jul 07 18:52:43.472777 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.472500 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c361ec7-0aef-4cf1-a191-28e4dc600802","Type":"ContainerDied","Data":"d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f"} Jul 07 18:52:43.472777 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.472542 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c361ec7-0aef-4cf1-a191-28e4dc600802","Type":"ContainerDied","Data":"16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f"} Jul 07 18:52:43.472777 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.472566 2564 scope.go:117] "RemoveContainer" containerID="7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d" Jul 07 18:52:43.472777 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.472582 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c361ec7-0aef-4cf1-a191-28e4dc600802","Type":"ContainerDied","Data":"d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4"} Jul 07 18:52:43.472777 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.472631 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c361ec7-0aef-4cf1-a191-28e4dc600802","Type":"ContainerDied","Data":"a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee"} Jul 07 18:52:43.472777 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.472672 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c361ec7-0aef-4cf1-a191-28e4dc600802","Type":"ContainerDied","Data":"b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39"} Jul 07 18:52:43.472777 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.472714 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5c361ec7-0aef-4cf1-a191-28e4dc600802","Type":"ContainerDied","Data":"c3360a15e8d450b70837c8c4031642b372bb30e3717a4390a9329189fc222bbc"} Jul 07 18:52:43.481502 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.481487 2564 scope.go:117] "RemoveContainer" containerID="d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f" Jul 07 18:52:43.489329 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.489306 2564 scope.go:117] "RemoveContainer" containerID="16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f" Jul 07 18:52:43.495840 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.495822 2564 scope.go:117] "RemoveContainer" containerID="d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4" Jul 07 18:52:43.502283 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.502268 2564 scope.go:117] "RemoveContainer" containerID="a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee" Jul 07 18:52:43.506829 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.506812 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:52:43.509315 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.509297 2564 scope.go:117] "RemoveContainer" containerID="b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39" Jul 07 18:52:43.513549 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.513529 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:52:43.516690 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.516673 2564 scope.go:117] "RemoveContainer" containerID="7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890" Jul 07 18:52:43.522934 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.522916 2564 scope.go:117] "RemoveContainer" containerID="7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d" Jul 07 18:52:43.523225 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:52:43.523176 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d\": container with ID starting with 7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d not found: ID does not exist" containerID="7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d" Jul 07 18:52:43.523282 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.523219 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d"} err="failed to get container status \"7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d\": rpc error: code = NotFound desc = could not find container \"7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d\": container with ID starting with 7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d not found: ID does not exist" Jul 07 18:52:43.523282 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.523239 2564 scope.go:117] "RemoveContainer" containerID="d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f" Jul 07 18:52:43.523471 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:52:43.523453 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f\": container with ID starting with d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f not found: ID does not exist" containerID="d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f" Jul 07 18:52:43.523532 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.523481 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f"} err="failed to get container status \"d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f\": rpc error: code = NotFound desc = could not find container \"d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f\": container with ID starting with d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f not found: ID does not exist" Jul 07 18:52:43.523532 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.523506 2564 scope.go:117] "RemoveContainer" containerID="16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f" Jul 07 18:52:43.523725 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:52:43.523710 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f\": container with ID starting with 16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f not found: ID does not exist" containerID="16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f" Jul 07 18:52:43.523770 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.523730 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f"} err="failed to get container status \"16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f\": rpc error: code = NotFound desc = could not find container \"16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f\": container with ID starting with 16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f not found: ID does not exist" Jul 07 18:52:43.523770 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.523744 2564 scope.go:117] "RemoveContainer" containerID="d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4" Jul 07 18:52:43.523964 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:52:43.523948 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4\": container with ID starting with d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4 not found: ID does not exist" containerID="d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4" Jul 07 18:52:43.524023 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.523971 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4"} err="failed to get container status \"d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4\": rpc error: code = NotFound desc = could not find container \"d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4\": container with ID starting with d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4 not found: ID does not exist" Jul 07 18:52:43.524023 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.523990 2564 scope.go:117] "RemoveContainer" containerID="a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee" Jul 07 18:52:43.524212 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:52:43.524184 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee\": container with ID starting with a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee not found: ID does not exist" containerID="a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee" Jul 07 18:52:43.524261 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.524217 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee"} err="failed to get container status \"a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee\": rpc error: code = NotFound desc = could not find container \"a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee\": container with ID starting with a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee not found: ID does not exist" Jul 07 18:52:43.524261 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.524230 2564 scope.go:117] "RemoveContainer" containerID="b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39" Jul 07 18:52:43.524461 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:52:43.524441 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39\": container with ID starting with b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39 not found: ID does not exist" containerID="b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39" Jul 07 18:52:43.524508 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.524468 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39"} err="failed to get container status \"b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39\": rpc error: code = NotFound desc = could not find container \"b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39\": container with ID starting with b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39 not found: ID does not exist" Jul 07 18:52:43.524508 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.524489 2564 scope.go:117] "RemoveContainer" containerID="7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890" Jul 07 18:52:43.524696 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:52:43.524678 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890\": container with ID starting with 7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890 not found: ID does not exist" containerID="7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890" Jul 07 18:52:43.524760 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.524704 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890"} err="failed to get container status \"7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890\": rpc error: code = NotFound desc = could not find container \"7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890\": container with ID starting with 7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890 not found: ID does not exist" Jul 07 18:52:43.524760 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.524724 2564 scope.go:117] "RemoveContainer" containerID="7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d" Jul 07 18:52:43.524933 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.524916 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d"} err="failed to get container status \"7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d\": rpc error: code = NotFound desc = could not find container \"7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d\": container with ID starting with 7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d not found: ID does not exist" Jul 07 18:52:43.524975 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.524934 2564 scope.go:117] "RemoveContainer" containerID="d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f" Jul 07 18:52:43.525131 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.525114 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f"} err="failed to get container status \"d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f\": rpc error: code = NotFound desc = could not find container \"d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f\": container with ID starting with d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f not found: ID does not exist" Jul 07 18:52:43.525210 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.525132 2564 scope.go:117] "RemoveContainer" containerID="16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f" Jul 07 18:52:43.525377 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.525353 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f"} err="failed to get container status \"16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f\": rpc error: code = NotFound desc = could not find container \"16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f\": container with ID starting with 16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f not found: ID does not exist" Jul 07 18:52:43.525422 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.525378 2564 scope.go:117] "RemoveContainer" containerID="d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4" Jul 07 18:52:43.525558 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.525542 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4"} err="failed to get container status \"d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4\": rpc error: code = NotFound desc = could not find container \"d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4\": container with ID starting with d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4 not found: ID does not exist" Jul 07 18:52:43.525608 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.525557 2564 scope.go:117] "RemoveContainer" containerID="a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee" Jul 07 18:52:43.525735 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.525717 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee"} err="failed to get container status \"a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee\": rpc error: code = NotFound desc = could not find container \"a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee\": container with ID starting with a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee not found: ID does not exist" Jul 07 18:52:43.525813 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.525736 2564 scope.go:117] "RemoveContainer" containerID="b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39" Jul 07 18:52:43.525958 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.525942 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39"} err="failed to get container status \"b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39\": rpc error: code = NotFound desc = could not find container \"b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39\": container with ID starting with b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39 not found: ID does not exist" Jul 07 18:52:43.526001 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.525957 2564 scope.go:117] "RemoveContainer" containerID="7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890" Jul 07 18:52:43.526161 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.526143 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890"} err="failed to get container status \"7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890\": rpc error: code = NotFound desc = could not find container \"7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890\": container with ID starting with 7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890 not found: ID does not exist" Jul 07 18:52:43.526258 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.526163 2564 scope.go:117] "RemoveContainer" containerID="7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d" Jul 07 18:52:43.526435 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.526419 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d"} err="failed to get container status \"7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d\": rpc error: code = NotFound desc = could not find container \"7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d\": container with ID starting with 7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d not found: ID does not exist" Jul 07 18:52:43.526489 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.526437 2564 scope.go:117] "RemoveContainer" containerID="d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f" Jul 07 18:52:43.526618 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.526603 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f"} err="failed to get container status \"d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f\": rpc error: code = NotFound desc = could not find container \"d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f\": container with ID starting with d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f not found: ID does not exist" Jul 07 18:52:43.526664 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.526618 2564 scope.go:117] "RemoveContainer" containerID="16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f" Jul 07 18:52:43.526782 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.526763 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f"} err="failed to get container status \"16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f\": rpc error: code = NotFound desc = could not find container \"16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f\": container with ID starting with 16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f not found: ID does not exist" Jul 07 18:52:43.526852 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.526784 2564 scope.go:117] "RemoveContainer" containerID="d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4" Jul 07 18:52:43.526989 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.526969 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4"} err="failed to get container status \"d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4\": rpc error: code = NotFound desc = could not find container \"d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4\": container with ID starting with d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4 not found: ID does not exist" Jul 07 18:52:43.527070 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.526991 2564 scope.go:117] "RemoveContainer" containerID="a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee" Jul 07 18:52:43.527142 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.527126 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee"} err="failed to get container status \"a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee\": rpc error: code = NotFound desc = could not find container \"a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee\": container with ID starting with a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee not found: ID does not exist" Jul 07 18:52:43.527215 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.527142 2564 scope.go:117] "RemoveContainer" containerID="b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39" Jul 07 18:52:43.527365 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.527351 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39"} err="failed to get container status \"b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39\": rpc error: code = NotFound desc = could not find container \"b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39\": container with ID starting with b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39 not found: ID does not exist" Jul 07 18:52:43.527417 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.527364 2564 scope.go:117] "RemoveContainer" containerID="7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890" Jul 07 18:52:43.527528 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.527511 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890"} err="failed to get container status \"7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890\": rpc error: code = NotFound desc = could not find container \"7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890\": container with ID starting with 7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890 not found: ID does not exist" Jul 07 18:52:43.527571 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.527528 2564 scope.go:117] "RemoveContainer" containerID="7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d" Jul 07 18:52:43.527701 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.527679 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d"} err="failed to get container status \"7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d\": rpc error: code = NotFound desc = could not find container \"7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d\": container with ID starting with 7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d not found: ID does not exist" Jul 07 18:52:43.527744 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.527703 2564 scope.go:117] "RemoveContainer" containerID="d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f" Jul 07 18:52:43.527893 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.527870 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f"} err="failed to get container status \"d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f\": rpc error: code = NotFound desc = could not find container \"d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f\": container with ID starting with d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f not found: ID does not exist" Jul 07 18:52:43.527935 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.527895 2564 scope.go:117] "RemoveContainer" containerID="16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f" Jul 07 18:52:43.528085 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.528067 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f"} err="failed to get container status \"16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f\": rpc error: code = NotFound desc = could not find container \"16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f\": container with ID starting with 16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f not found: ID does not exist" Jul 07 18:52:43.528127 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.528086 2564 scope.go:117] "RemoveContainer" containerID="d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4" Jul 07 18:52:43.528329 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.528312 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4"} err="failed to get container status \"d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4\": rpc error: code = NotFound desc = could not find container \"d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4\": container with ID starting with d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4 not found: ID does not exist" Jul 07 18:52:43.528329 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.528329 2564 scope.go:117] "RemoveContainer" containerID="a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee" Jul 07 18:52:43.528594 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.528569 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee"} err="failed to get container status \"a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee\": rpc error: code = NotFound desc = could not find container \"a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee\": container with ID starting with a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee not found: ID does not exist" Jul 07 18:52:43.528594 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.528595 2564 scope.go:117] "RemoveContainer" containerID="b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39" Jul 07 18:52:43.528858 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.528833 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39"} err="failed to get container status \"b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39\": rpc error: code = NotFound desc = could not find container \"b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39\": container with ID starting with b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39 not found: ID does not exist" Jul 07 18:52:43.528952 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.528860 2564 scope.go:117] "RemoveContainer" containerID="7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890" Jul 07 18:52:43.529180 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.529158 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890"} err="failed to get container status \"7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890\": rpc error: code = NotFound desc = could not find container \"7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890\": container with ID starting with 7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890 not found: ID does not exist" Jul 07 18:52:43.529285 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.529181 2564 scope.go:117] "RemoveContainer" containerID="7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d" Jul 07 18:52:43.529409 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.529392 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d"} err="failed to get container status \"7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d\": rpc error: code = NotFound desc = could not find container \"7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d\": container with ID starting with 7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d not found: ID does not exist" Jul 07 18:52:43.529474 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.529410 2564 scope.go:117] "RemoveContainer" containerID="d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f" Jul 07 18:52:43.529598 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.529584 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f"} err="failed to get container status \"d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f\": rpc error: code = NotFound desc = could not find container \"d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f\": container with ID starting with d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f not found: ID does not exist" Jul 07 18:52:43.529638 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.529597 2564 scope.go:117] "RemoveContainer" containerID="16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f" Jul 07 18:52:43.529766 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.529751 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f"} err="failed to get container status \"16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f\": rpc error: code = NotFound desc = could not find container \"16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f\": container with ID starting with 16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f not found: ID does not exist" Jul 07 18:52:43.529815 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.529765 2564 scope.go:117] "RemoveContainer" containerID="d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4" Jul 07 18:52:43.529944 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.529927 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4"} err="failed to get container status \"d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4\": rpc error: code = NotFound desc = could not find container \"d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4\": container with ID starting with d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4 not found: ID does not exist" Jul 07 18:52:43.529986 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.529945 2564 scope.go:117] "RemoveContainer" containerID="a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee" Jul 07 18:52:43.530115 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.530099 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee"} err="failed to get container status \"a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee\": rpc error: code = NotFound desc = could not find container \"a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee\": container with ID starting with a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee not found: ID does not exist" Jul 07 18:52:43.530156 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.530115 2564 scope.go:117] "RemoveContainer" containerID="b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39" Jul 07 18:52:43.530327 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.530308 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39"} err="failed to get container status \"b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39\": rpc error: code = NotFound desc = could not find container \"b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39\": container with ID starting with b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39 not found: ID does not exist" Jul 07 18:52:43.530410 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.530328 2564 scope.go:117] "RemoveContainer" containerID="7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890" Jul 07 18:52:43.530550 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.530532 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890"} err="failed to get container status \"7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890\": rpc error: code = NotFound desc = could not find container \"7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890\": container with ID starting with 7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890 not found: ID does not exist" Jul 07 18:52:43.530650 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.530549 2564 scope.go:117] "RemoveContainer" containerID="7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d" Jul 07 18:52:43.530800 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.530782 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d"} err="failed to get container status \"7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d\": rpc error: code = NotFound desc = could not find container \"7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d\": container with ID starting with 7d75253faf67bf6b99d8e0f14d8e08865041881382f95f101c57235accb1504d not found: ID does not exist" Jul 07 18:52:43.530848 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.530801 2564 scope.go:117] "RemoveContainer" containerID="d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f" Jul 07 18:52:43.531020 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.531002 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f"} err="failed to get container status \"d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f\": rpc error: code = NotFound desc = could not find container \"d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f\": container with ID starting with d3a66987eb9bbeb11a6c433bfe16f7016714c90a61be9348dd2efbf733a8b02f not found: ID does not exist" Jul 07 18:52:43.531071 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.531021 2564 scope.go:117] "RemoveContainer" containerID="16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f" Jul 07 18:52:43.531262 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.531245 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f"} err="failed to get container status \"16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f\": rpc error: code = NotFound desc = could not find container \"16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f\": container with ID starting with 16203ebd98f0093a0b05c6c9edc708b99ccd4afcaf09075d47c07ba2b8fc428f not found: ID does not exist" Jul 07 18:52:43.531314 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.531261 2564 scope.go:117] "RemoveContainer" containerID="d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4" Jul 07 18:52:43.531488 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.531471 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4"} err="failed to get container status \"d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4\": rpc error: code = NotFound desc = could not find container \"d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4\": container with ID starting with d3e70d4deca1222db435a87d62ae2fe0c230227c67e2fe5a7ddf7e45d26566c4 not found: ID does not exist" Jul 07 18:52:43.531488 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.531487 2564 scope.go:117] "RemoveContainer" containerID="a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee" Jul 07 18:52:43.531675 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.531660 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee"} err="failed to get container status \"a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee\": rpc error: code = NotFound desc = could not find container \"a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee\": container with ID starting with a0d9d03f465e2509867aa73459b164d60bf4a20a5525fe8dfb23527be6ebbdee not found: ID does not exist" Jul 07 18:52:43.531713 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.531676 2564 scope.go:117] "RemoveContainer" containerID="b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39" Jul 07 18:52:43.531849 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.531830 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39"} err="failed to get container status \"b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39\": rpc error: code = NotFound desc = could not find container \"b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39\": container with ID starting with b89263f80d0ccbe3204ce316e6767e4794aeda8088bc4525ef9d72d0e2408a39 not found: ID does not exist" Jul 07 18:52:43.531891 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.531851 2564 scope.go:117] "RemoveContainer" containerID="7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890" Jul 07 18:52:43.532058 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.532034 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890"} err="failed to get container status \"7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890\": rpc error: code = NotFound desc = could not find container \"7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890\": container with ID starting with 7c8a7ef585f15846ab2efcea7ca7b89e0d650ba267196ce80c701f5a0b2ab890 not found: ID does not exist" Jul 07 18:52:43.542501 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542481 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:52:43.542764 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542752 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="init-config-reloader" Jul 07 18:52:43.542802 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542766 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="init-config-reloader" Jul 07 18:52:43.542802 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542787 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d720eaf5-7915-48b7-8f9b-8b3e7eaaec46" containerName="console" Jul 07 18:52:43.542802 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542793 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d720eaf5-7915-48b7-8f9b-8b3e7eaaec46" containerName="console" Jul 07 18:52:43.542802 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542800 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="config-reloader" Jul 07 18:52:43.542917 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542806 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="config-reloader" Jul 07 18:52:43.542917 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542812 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="prometheus" Jul 07 18:52:43.542917 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542817 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="prometheus" Jul 07 18:52:43.542917 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542829 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="kube-rbac-proxy" Jul 07 18:52:43.542917 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542834 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="kube-rbac-proxy" Jul 07 18:52:43.542917 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542844 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="thanos-sidecar" Jul 07 18:52:43.542917 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542849 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="thanos-sidecar" Jul 07 18:52:43.542917 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542861 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="kube-rbac-proxy-thanos" Jul 07 18:52:43.542917 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542866 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="kube-rbac-proxy-thanos" Jul 07 18:52:43.542917 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542871 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="kube-rbac-proxy-web" Jul 07 18:52:43.542917 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542876 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="kube-rbac-proxy-web" Jul 07 18:52:43.542917 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542882 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe8ffd85-f400-45e4-b88d-457e745d9132" containerName="console" Jul 07 18:52:43.542917 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542887 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe8ffd85-f400-45e4-b88d-457e745d9132" containerName="console" Jul 07 18:52:43.543338 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542942 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="prometheus" Jul 07 18:52:43.543338 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542949 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="config-reloader" Jul 07 18:52:43.543338 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542955 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="kube-rbac-proxy-web" Jul 07 18:52:43.543338 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542961 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="kube-rbac-proxy" Jul 07 18:52:43.543338 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542966 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="thanos-sidecar" Jul 07 18:52:43.543338 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542971 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" containerName="kube-rbac-proxy-thanos" Jul 07 18:52:43.543338 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542976 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe8ffd85-f400-45e4-b88d-457e745d9132" containerName="console" Jul 07 18:52:43.543338 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.542985 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d720eaf5-7915-48b7-8f9b-8b3e7eaaec46" containerName="console" Jul 07 18:52:43.552118 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.552096 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.554575 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.554551 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 18:52:43.554575 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.554550 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 18:52:43.554735 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.554586 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 18:52:43.554735 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.554676 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 18:52:43.554837 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.554761 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 18:52:43.554837 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.554784 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 18:52:43.555144 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.555124 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 18:52:43.555273 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.555167 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 18:52:43.555273 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.555184 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 18:52:43.555414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.555300 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 18:52:43.555583 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.555567 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pzq75\"" Jul 07 18:52:43.555651 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.555604 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 18:52:43.555711 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.555608 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 18:52:43.561418 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.561386 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 18:52:43.561714 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.561690 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2nnonr638kt8q\"" Jul 07 18:52:43.575928 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.575905 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 18:52:43.577852 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.577833 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:52:43.587746 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.587728 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 18:52:43.636679 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.636655 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.636784 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.636682 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-web-config\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.636784 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.636699 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/90e33b63-2861-47a3-ad96-e6aff8096bda-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.636784 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.636717 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d4bn\" (UniqueName: \"kubernetes.io/projected/90e33b63-2861-47a3-ad96-e6aff8096bda-kube-api-access-6d4bn\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.636890 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.636792 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90e33b63-2861-47a3-ad96-e6aff8096bda-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.636890 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.636818 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.636890 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.636844 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/90e33b63-2861-47a3-ad96-e6aff8096bda-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.636890 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.636861 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.636890 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.636882 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.637126 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.636914 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90e33b63-2861-47a3-ad96-e6aff8096bda-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.637126 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.636935 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/90e33b63-2861-47a3-ad96-e6aff8096bda-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.637126 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.636993 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/90e33b63-2861-47a3-ad96-e6aff8096bda-config-out\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.637126 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.637013 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.637126 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.637029 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90e33b63-2861-47a3-ad96-e6aff8096bda-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.637126 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.637051 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/90e33b63-2861-47a3-ad96-e6aff8096bda-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.637126 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.637078 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-config\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.637410 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.637138 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.637410 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.637178 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/90e33b63-2861-47a3-ad96-e6aff8096bda-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.637410 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.637225 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.637410 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.637243 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90e33b63-2861-47a3-ad96-e6aff8096bda-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.738666 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.738596 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.738666 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.738629 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/90e33b63-2861-47a3-ad96-e6aff8096bda-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.738666 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.738647 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.739182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.738671 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.739182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.738698 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90e33b63-2861-47a3-ad96-e6aff8096bda-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.739182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.738722 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/90e33b63-2861-47a3-ad96-e6aff8096bda-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.739182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.738752 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/90e33b63-2861-47a3-ad96-e6aff8096bda-config-out\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.739182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.738777 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.739182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.738803 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90e33b63-2861-47a3-ad96-e6aff8096bda-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.739182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.738842 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/90e33b63-2861-47a3-ad96-e6aff8096bda-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.739182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.738876 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-config\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.739182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.738900 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.739182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.738926 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/90e33b63-2861-47a3-ad96-e6aff8096bda-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.739182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.738960 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.739182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.738986 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90e33b63-2861-47a3-ad96-e6aff8096bda-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.739182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.739027 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.739182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.739056 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-web-config\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.739901 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.739084 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/90e33b63-2861-47a3-ad96-e6aff8096bda-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.739901 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.739111 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d4bn\" (UniqueName: \"kubernetes.io/projected/90e33b63-2861-47a3-ad96-e6aff8096bda-kube-api-access-6d4bn\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.739901 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.739160 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90e33b63-2861-47a3-ad96-e6aff8096bda-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.739901 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.739522 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/90e33b63-2861-47a3-ad96-e6aff8096bda-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.739901 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.739578 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90e33b63-2861-47a3-ad96-e6aff8096bda-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.739901 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.739724 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/90e33b63-2861-47a3-ad96-e6aff8096bda-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.740218 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.739989 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90e33b63-2861-47a3-ad96-e6aff8096bda-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.740286 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.740230 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90e33b63-2861-47a3-ad96-e6aff8096bda-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.741154 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.740824 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/90e33b63-2861-47a3-ad96-e6aff8096bda-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.741997 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.741748 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.741997 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.741901 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.741997 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.741932 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/90e33b63-2861-47a3-ad96-e6aff8096bda-config-out\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.742573 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.742548 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.742676 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.742590 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/90e33b63-2861-47a3-ad96-e6aff8096bda-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.742676 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.742636 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.743023 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.742999 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/90e33b63-2861-47a3-ad96-e6aff8096bda-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.743123 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.742998 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.743401 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.743384 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-config\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.744727 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.744698 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-web-config\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.745113 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.745095 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/90e33b63-2861-47a3-ad96-e6aff8096bda-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.745236 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.745213 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.745313 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.745296 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/90e33b63-2861-47a3-ad96-e6aff8096bda-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.752002 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.751974 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d4bn\" (UniqueName: \"kubernetes.io/projected/90e33b63-2861-47a3-ad96-e6aff8096bda-kube-api-access-6d4bn\") pod \"prometheus-k8s-0\" (UID: \"90e33b63-2861-47a3-ad96-e6aff8096bda\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:43.862413 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:43.862384 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:44.004130 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:44.004108 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 18:52:44.007947 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:52:44.007919 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90e33b63_2861_47a3_ad96_e6aff8096bda.slice/crio-d3ed60dfb9083a227c2b9f08d8a9dee576f60b4f25bc67fde9a89362f65b70d1 WatchSource:0}: Error finding container d3ed60dfb9083a227c2b9f08d8a9dee576f60b4f25bc67fde9a89362f65b70d1: Status 404 returned error can't find the container with id d3ed60dfb9083a227c2b9f08d8a9dee576f60b4f25bc67fde9a89362f65b70d1 Jul 07 18:52:44.477856 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:44.477824 2564 generic.go:356] "Generic (PLEG): container finished" podID="90e33b63-2861-47a3-ad96-e6aff8096bda" containerID="f1309c94e120e846520fbf2f89fa6a353ec4b88a051baeaf4ffa85ab5ec6dd8a" exitCode=0 Jul 07 18:52:44.478020 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:44.477914 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90e33b63-2861-47a3-ad96-e6aff8096bda","Type":"ContainerDied","Data":"f1309c94e120e846520fbf2f89fa6a353ec4b88a051baeaf4ffa85ab5ec6dd8a"} Jul 07 18:52:44.478020 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:44.478002 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90e33b63-2861-47a3-ad96-e6aff8096bda","Type":"ContainerStarted","Data":"d3ed60dfb9083a227c2b9f08d8a9dee576f60b4f25bc67fde9a89362f65b70d1"} Jul 07 18:52:44.573928 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:44.573899 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c361ec7-0aef-4cf1-a191-28e4dc600802" path="/var/lib/kubelet/pods/5c361ec7-0aef-4cf1-a191-28e4dc600802/volumes" Jul 07 18:52:45.487135 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:45.487102 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90e33b63-2861-47a3-ad96-e6aff8096bda","Type":"ContainerStarted","Data":"7f607686e3fc60236cd0bdfc8cb05972e5d8a066bac58a7c02def5e7a96f1522"} Jul 07 18:52:45.487546 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:45.487165 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90e33b63-2861-47a3-ad96-e6aff8096bda","Type":"ContainerStarted","Data":"1cf042a59de2f541ffa6d178899bb07f65b145e0b31ff023529cc2b093e6b3e2"} Jul 07 18:52:45.487546 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:45.487191 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90e33b63-2861-47a3-ad96-e6aff8096bda","Type":"ContainerStarted","Data":"50d3cf4f3501e8652c038ec38617d6a38cba5343913a79dee8231a1629888173"} Jul 07 18:52:45.487546 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:45.487232 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90e33b63-2861-47a3-ad96-e6aff8096bda","Type":"ContainerStarted","Data":"73179bac6adc761a83c924b31340c25864b52049a8f72361d6e74d9ae0c88e51"} Jul 07 18:52:45.487546 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:45.487255 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90e33b63-2861-47a3-ad96-e6aff8096bda","Type":"ContainerStarted","Data":"31df5c920d95305d9db2bbb43fa18d2234e28f95f2cf919c223ac87ecbf102ca"} Jul 07 18:52:45.487546 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:45.487277 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"90e33b63-2861-47a3-ad96-e6aff8096bda","Type":"ContainerStarted","Data":"76b46aeb816756a80668d0882accb3890b396bc5241f62d817cd743a7a107a85"} Jul 07 18:52:45.530464 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:45.530393 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.509438046 podStartE2EDuration="2.530376002s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.937956ms" isStatefulPod=false podCreationTimestamp="2026-07-07 18:52:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:52:45.528090844 +0000 UTC m=+177.497726883" watchObservedRunningTime="2026-07-07 18:52:45.530376002 +0000 UTC m=+177.500012019" Jul 07 18:52:48.863397 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:48.863362 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:52:57.406711 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:52:57.406676 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f8d9549fc-b88tb"] Jul 07 18:53:22.431752 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.431706 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5f8d9549fc-b88tb" podUID="ca83df20-4431-4e6c-bf97-00861641ad97" containerName="console" containerID="cri-o://dff112c05d2ef4e143cac927f1c4bfeb653abbea279d2b94405a86d869d86870" gracePeriod=15 Jul 07 18:53:22.639167 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.639138 2564 generic.go:356] "Generic (PLEG): container finished" podID="ca83df20-4431-4e6c-bf97-00861641ad97" containerID="dff112c05d2ef4e143cac927f1c4bfeb653abbea279d2b94405a86d869d86870" exitCode=0 Jul 07 18:53:22.639345 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.639220 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f8d9549fc-b88tb" event={"ID":"ca83df20-4431-4e6c-bf97-00861641ad97","Type":"ContainerDied","Data":"dff112c05d2ef4e143cac927f1c4bfeb653abbea279d2b94405a86d869d86870"} Jul 07 18:53:22.675115 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.675096 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:53:22.756061 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.756038 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca83df20-4431-4e6c-bf97-00861641ad97-console-serving-cert\") pod \"ca83df20-4431-4e6c-bf97-00861641ad97\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " Jul 07 18:53:22.756231 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.756093 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-oauth-serving-cert\") pod \"ca83df20-4431-4e6c-bf97-00861641ad97\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " Jul 07 18:53:22.756316 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.756268 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca83df20-4431-4e6c-bf97-00861641ad97-console-oauth-config\") pod \"ca83df20-4431-4e6c-bf97-00861641ad97\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " Jul 07 18:53:22.756316 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.756306 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-px5vn\" (UniqueName: \"kubernetes.io/projected/ca83df20-4431-4e6c-bf97-00861641ad97-kube-api-access-px5vn\") pod \"ca83df20-4431-4e6c-bf97-00861641ad97\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " Jul 07 18:53:22.756428 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.756341 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-console-config\") pod \"ca83df20-4431-4e6c-bf97-00861641ad97\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " Jul 07 18:53:22.756428 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.756356 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-service-ca\") pod \"ca83df20-4431-4e6c-bf97-00861641ad97\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " Jul 07 18:53:22.756428 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.756386 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-trusted-ca-bundle\") pod \"ca83df20-4431-4e6c-bf97-00861641ad97\" (UID: \"ca83df20-4431-4e6c-bf97-00861641ad97\") " Jul 07 18:53:22.756858 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.756487 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ca83df20-4431-4e6c-bf97-00861641ad97" (UID: "ca83df20-4431-4e6c-bf97-00861641ad97"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:53:22.756858 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.756669 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-oauth-serving-cert\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:53:22.756858 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.756804 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-console-config" (OuterVolumeSpecName: "console-config") pod "ca83df20-4431-4e6c-bf97-00861641ad97" (UID: "ca83df20-4431-4e6c-bf97-00861641ad97"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:53:22.756858 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.756811 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-service-ca" (OuterVolumeSpecName: "service-ca") pod "ca83df20-4431-4e6c-bf97-00861641ad97" (UID: "ca83df20-4431-4e6c-bf97-00861641ad97"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:53:22.757021 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.756889 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ca83df20-4431-4e6c-bf97-00861641ad97" (UID: "ca83df20-4431-4e6c-bf97-00861641ad97"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:53:22.758256 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.758236 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca83df20-4431-4e6c-bf97-00861641ad97-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ca83df20-4431-4e6c-bf97-00861641ad97" (UID: "ca83df20-4431-4e6c-bf97-00861641ad97"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:53:22.758322 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.758264 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca83df20-4431-4e6c-bf97-00861641ad97-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ca83df20-4431-4e6c-bf97-00861641ad97" (UID: "ca83df20-4431-4e6c-bf97-00861641ad97"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:53:22.758410 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.758391 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca83df20-4431-4e6c-bf97-00861641ad97-kube-api-access-px5vn" (OuterVolumeSpecName: "kube-api-access-px5vn") pod "ca83df20-4431-4e6c-bf97-00861641ad97" (UID: "ca83df20-4431-4e6c-bf97-00861641ad97"). InnerVolumeSpecName "kube-api-access-px5vn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:53:22.857939 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.857915 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca83df20-4431-4e6c-bf97-00861641ad97-console-oauth-config\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:53:22.857939 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.857937 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-px5vn\" (UniqueName: \"kubernetes.io/projected/ca83df20-4431-4e6c-bf97-00861641ad97-kube-api-access-px5vn\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:53:22.858075 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.857947 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-console-config\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:53:22.858075 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.857956 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-service-ca\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:53:22.858075 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.857963 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ca83df20-4431-4e6c-bf97-00861641ad97-trusted-ca-bundle\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:53:22.858075 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:22.857971 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca83df20-4431-4e6c-bf97-00861641ad97-console-serving-cert\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:53:23.644304 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:23.644274 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f8d9549fc-b88tb" Jul 07 18:53:23.644749 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:23.644274 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f8d9549fc-b88tb" event={"ID":"ca83df20-4431-4e6c-bf97-00861641ad97","Type":"ContainerDied","Data":"0232a3cdb394d3a13cc23ce97b3c3f182fde0218c1d4113d6bc17793183bbe5f"} Jul 07 18:53:23.644749 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:23.644404 2564 scope.go:117] "RemoveContainer" containerID="dff112c05d2ef4e143cac927f1c4bfeb653abbea279d2b94405a86d869d86870" Jul 07 18:53:23.668346 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:23.668317 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f8d9549fc-b88tb"] Jul 07 18:53:23.674992 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:23.674972 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f8d9549fc-b88tb"] Jul 07 18:53:24.575071 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:24.575040 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca83df20-4431-4e6c-bf97-00861641ad97" path="/var/lib/kubelet/pods/ca83df20-4431-4e6c-bf97-00861641ad97/volumes" Jul 07 18:53:43.862836 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:43.862801 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:53:43.877858 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:43.877835 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:53:44.741378 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:44.741352 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 18:53:56.737756 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:56.737723 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-6s9qm"] Jul 07 18:53:56.738316 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:56.738037 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca83df20-4431-4e6c-bf97-00861641ad97" containerName="console" Jul 07 18:53:56.738316 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:56.738049 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca83df20-4431-4e6c-bf97-00861641ad97" containerName="console" Jul 07 18:53:56.738316 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:56.738119 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca83df20-4431-4e6c-bf97-00861641ad97" containerName="console" Jul 07 18:53:56.742305 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:56.742285 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-6s9qm" Jul 07 18:53:56.744523 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:56.744502 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 18:53:56.745399 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:56.745382 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-t62xk\"" Jul 07 18:53:56.745399 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:56.745392 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 18:53:56.754734 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:56.754709 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-6s9qm"] Jul 07 18:53:56.816420 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:56.816390 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxszv\" (UniqueName: \"kubernetes.io/projected/23b05cf1-99b0-4e4e-badd-ce46de35a123-kube-api-access-wxszv\") pod \"cert-manager-webhook-6486ffd997-6s9qm\" (UID: \"23b05cf1-99b0-4e4e-badd-ce46de35a123\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6s9qm" Jul 07 18:53:56.816532 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:56.816441 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23b05cf1-99b0-4e4e-badd-ce46de35a123-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-6s9qm\" (UID: \"23b05cf1-99b0-4e4e-badd-ce46de35a123\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6s9qm" Jul 07 18:53:56.916835 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:56.916811 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23b05cf1-99b0-4e4e-badd-ce46de35a123-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-6s9qm\" (UID: \"23b05cf1-99b0-4e4e-badd-ce46de35a123\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6s9qm" Jul 07 18:53:56.916968 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:56.916868 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxszv\" (UniqueName: \"kubernetes.io/projected/23b05cf1-99b0-4e4e-badd-ce46de35a123-kube-api-access-wxszv\") pod \"cert-manager-webhook-6486ffd997-6s9qm\" (UID: \"23b05cf1-99b0-4e4e-badd-ce46de35a123\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6s9qm" Jul 07 18:53:56.928878 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:56.928856 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxszv\" (UniqueName: \"kubernetes.io/projected/23b05cf1-99b0-4e4e-badd-ce46de35a123-kube-api-access-wxszv\") pod \"cert-manager-webhook-6486ffd997-6s9qm\" (UID: \"23b05cf1-99b0-4e4e-badd-ce46de35a123\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6s9qm" Jul 07 18:53:56.928969 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:56.928881 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/23b05cf1-99b0-4e4e-badd-ce46de35a123-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-6s9qm\" (UID: \"23b05cf1-99b0-4e4e-badd-ce46de35a123\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6s9qm" Jul 07 18:53:57.066892 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:57.066821 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-6s9qm" Jul 07 18:53:57.197265 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:57.197141 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-6s9qm"] Jul 07 18:53:57.199773 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:53:57.199745 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23b05cf1_99b0_4e4e_badd_ce46de35a123.slice/crio-7cd3b7e14d4650ec1e7834f362171beb10ec05d06efa0807ef489ca1dfdafed8 WatchSource:0}: Error finding container 7cd3b7e14d4650ec1e7834f362171beb10ec05d06efa0807ef489ca1dfdafed8: Status 404 returned error can't find the container with id 7cd3b7e14d4650ec1e7834f362171beb10ec05d06efa0807ef489ca1dfdafed8 Jul 07 18:53:57.778510 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:53:57.778470 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-6s9qm" event={"ID":"23b05cf1-99b0-4e4e-badd-ce46de35a123","Type":"ContainerStarted","Data":"7cd3b7e14d4650ec1e7834f362171beb10ec05d06efa0807ef489ca1dfdafed8"} Jul 07 18:54:00.793083 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:00.793046 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-6s9qm" event={"ID":"23b05cf1-99b0-4e4e-badd-ce46de35a123","Type":"ContainerStarted","Data":"116019fbe479518e684dd91a400524fa0c6bff46b2c417064c68a4abaa265224"} Jul 07 18:54:00.793484 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:00.793222 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-6s9qm" Jul 07 18:54:00.813452 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:00.813407 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-6s9qm" podStartSLOduration=1.944723929 podStartE2EDuration="4.813392972s" totalImagesPullingTime="2.868669043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:53:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:54:00.81253947 +0000 UTC m=+252.782175514" watchObservedRunningTime="2026-07-07 18:54:00.813392972 +0000 UTC m=+252.783028989" Jul 07 18:54:06.799025 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:06.798995 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-6s9qm" Jul 07 18:54:29.949546 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:29.949475 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt"] Jul 07 18:54:30.007425 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.007384 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt"] Jul 07 18:54:30.007584 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.007492 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt" Jul 07 18:54:30.010626 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.010604 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 18:54:30.011503 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.011481 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-9sm6b\"" Jul 07 18:54:30.011626 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.011489 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 18:54:30.011626 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.011489 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 18:54:30.011626 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.011540 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 18:54:30.011787 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.011660 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 18:54:30.091057 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.091037 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db38a546-4d03-4882-942c-3da5e96a7400-cert\") pod \"lws-controller-manager-6d5b9d7dd9-kbtbt\" (UID: \"db38a546-4d03-4882-942c-3da5e96a7400\") " pod="openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt" Jul 07 18:54:30.091180 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.091069 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/db38a546-4d03-4882-942c-3da5e96a7400-manager-config\") pod \"lws-controller-manager-6d5b9d7dd9-kbtbt\" (UID: \"db38a546-4d03-4882-942c-3da5e96a7400\") " pod="openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt" Jul 07 18:54:30.091180 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.091113 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/db38a546-4d03-4882-942c-3da5e96a7400-metrics-cert\") pod \"lws-controller-manager-6d5b9d7dd9-kbtbt\" (UID: \"db38a546-4d03-4882-942c-3da5e96a7400\") " pod="openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt" Jul 07 18:54:30.091180 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.091136 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd8r9\" (UniqueName: \"kubernetes.io/projected/db38a546-4d03-4882-942c-3da5e96a7400-kube-api-access-xd8r9\") pod \"lws-controller-manager-6d5b9d7dd9-kbtbt\" (UID: \"db38a546-4d03-4882-942c-3da5e96a7400\") " pod="openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt" Jul 07 18:54:30.191823 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.191788 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db38a546-4d03-4882-942c-3da5e96a7400-cert\") pod \"lws-controller-manager-6d5b9d7dd9-kbtbt\" (UID: \"db38a546-4d03-4882-942c-3da5e96a7400\") " pod="openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt" Jul 07 18:54:30.191979 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.191835 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/db38a546-4d03-4882-942c-3da5e96a7400-manager-config\") pod \"lws-controller-manager-6d5b9d7dd9-kbtbt\" (UID: \"db38a546-4d03-4882-942c-3da5e96a7400\") " pod="openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt" Jul 07 18:54:30.191979 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.191899 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/db38a546-4d03-4882-942c-3da5e96a7400-metrics-cert\") pod \"lws-controller-manager-6d5b9d7dd9-kbtbt\" (UID: \"db38a546-4d03-4882-942c-3da5e96a7400\") " pod="openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt" Jul 07 18:54:30.191979 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.191923 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd8r9\" (UniqueName: \"kubernetes.io/projected/db38a546-4d03-4882-942c-3da5e96a7400-kube-api-access-xd8r9\") pod \"lws-controller-manager-6d5b9d7dd9-kbtbt\" (UID: \"db38a546-4d03-4882-942c-3da5e96a7400\") " pod="openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt" Jul 07 18:54:30.192552 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.192528 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/db38a546-4d03-4882-942c-3da5e96a7400-manager-config\") pod \"lws-controller-manager-6d5b9d7dd9-kbtbt\" (UID: \"db38a546-4d03-4882-942c-3da5e96a7400\") " pod="openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt" Jul 07 18:54:30.194332 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.194305 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/db38a546-4d03-4882-942c-3da5e96a7400-metrics-cert\") pod \"lws-controller-manager-6d5b9d7dd9-kbtbt\" (UID: \"db38a546-4d03-4882-942c-3da5e96a7400\") " pod="openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt" Jul 07 18:54:30.194491 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.194471 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/db38a546-4d03-4882-942c-3da5e96a7400-cert\") pod \"lws-controller-manager-6d5b9d7dd9-kbtbt\" (UID: \"db38a546-4d03-4882-942c-3da5e96a7400\") " pod="openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt" Jul 07 18:54:30.202154 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.202103 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd8r9\" (UniqueName: \"kubernetes.io/projected/db38a546-4d03-4882-942c-3da5e96a7400-kube-api-access-xd8r9\") pod \"lws-controller-manager-6d5b9d7dd9-kbtbt\" (UID: \"db38a546-4d03-4882-942c-3da5e96a7400\") " pod="openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt" Jul 07 18:54:30.317245 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.317212 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt" Jul 07 18:54:30.472254 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.472233 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt"] Jul 07 18:54:30.474399 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:54:30.474370 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb38a546_4d03_4882_942c_3da5e96a7400.slice/crio-90a0093549ceb4b8d569620c22a0c667eb228eac170d0be6a108904629c18f48 WatchSource:0}: Error finding container 90a0093549ceb4b8d569620c22a0c667eb228eac170d0be6a108904629c18f48: Status 404 returned error can't find the container with id 90a0093549ceb4b8d569620c22a0c667eb228eac170d0be6a108904629c18f48 Jul 07 18:54:30.483934 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.483913 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr"] Jul 07 18:54:30.517248 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.517233 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" Jul 07 18:54:30.522191 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.522170 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 18:54:30.522640 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.522624 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 18:54:30.525077 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.525061 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 18:54:30.536907 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.536892 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 18:54:30.553112 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.553091 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr"] Jul 07 18:54:30.564344 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.564327 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q8s9r\"" Jul 07 18:54:30.595796 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.595771 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f7c7b22-3588-4137-ba9f-2dd8d8889ae6-tmp\") pod \"opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr\" (UID: \"5f7c7b22-3588-4137-ba9f-2dd8d8889ae6\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" Jul 07 18:54:30.595902 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.595801 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5f7c7b22-3588-4137-ba9f-2dd8d8889ae6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr\" (UID: \"5f7c7b22-3588-4137-ba9f-2dd8d8889ae6\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" Jul 07 18:54:30.595902 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.595838 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5f7c7b22-3588-4137-ba9f-2dd8d8889ae6-manifests\") pod \"opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr\" (UID: \"5f7c7b22-3588-4137-ba9f-2dd8d8889ae6\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" Jul 07 18:54:30.595902 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.595895 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f7c7b22-3588-4137-ba9f-2dd8d8889ae6-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr\" (UID: \"5f7c7b22-3588-4137-ba9f-2dd8d8889ae6\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" Jul 07 18:54:30.596072 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.595921 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnzjg\" (UniqueName: \"kubernetes.io/projected/5f7c7b22-3588-4137-ba9f-2dd8d8889ae6-kube-api-access-pnzjg\") pod \"opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr\" (UID: \"5f7c7b22-3588-4137-ba9f-2dd8d8889ae6\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" Jul 07 18:54:30.696660 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.696635 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f7c7b22-3588-4137-ba9f-2dd8d8889ae6-tmp\") pod \"opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr\" (UID: \"5f7c7b22-3588-4137-ba9f-2dd8d8889ae6\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" Jul 07 18:54:30.696748 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.696663 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5f7c7b22-3588-4137-ba9f-2dd8d8889ae6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr\" (UID: \"5f7c7b22-3588-4137-ba9f-2dd8d8889ae6\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" Jul 07 18:54:30.696748 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.696694 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5f7c7b22-3588-4137-ba9f-2dd8d8889ae6-manifests\") pod \"opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr\" (UID: \"5f7c7b22-3588-4137-ba9f-2dd8d8889ae6\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" Jul 07 18:54:30.696853 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.696829 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f7c7b22-3588-4137-ba9f-2dd8d8889ae6-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr\" (UID: \"5f7c7b22-3588-4137-ba9f-2dd8d8889ae6\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" Jul 07 18:54:30.696964 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.696896 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnzjg\" (UniqueName: \"kubernetes.io/projected/5f7c7b22-3588-4137-ba9f-2dd8d8889ae6-kube-api-access-pnzjg\") pod \"opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr\" (UID: \"5f7c7b22-3588-4137-ba9f-2dd8d8889ae6\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" Jul 07 18:54:30.697047 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.697024 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5f7c7b22-3588-4137-ba9f-2dd8d8889ae6-manifests\") pod \"opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr\" (UID: \"5f7c7b22-3588-4137-ba9f-2dd8d8889ae6\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" Jul 07 18:54:30.698956 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.698931 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5f7c7b22-3588-4137-ba9f-2dd8d8889ae6-tmp\") pod \"opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr\" (UID: \"5f7c7b22-3588-4137-ba9f-2dd8d8889ae6\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" Jul 07 18:54:30.699159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.699141 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5f7c7b22-3588-4137-ba9f-2dd8d8889ae6-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr\" (UID: \"5f7c7b22-3588-4137-ba9f-2dd8d8889ae6\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" Jul 07 18:54:30.699224 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.699158 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5f7c7b22-3588-4137-ba9f-2dd8d8889ae6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr\" (UID: \"5f7c7b22-3588-4137-ba9f-2dd8d8889ae6\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" Jul 07 18:54:30.721696 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.721651 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnzjg\" (UniqueName: \"kubernetes.io/projected/5f7c7b22-3588-4137-ba9f-2dd8d8889ae6-kube-api-access-pnzjg\") pod \"opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr\" (UID: \"5f7c7b22-3588-4137-ba9f-2dd8d8889ae6\") " pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" Jul 07 18:54:30.828275 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.828254 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" Jul 07 18:54:30.905600 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.905562 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt" event={"ID":"db38a546-4d03-4882-942c-3da5e96a7400","Type":"ContainerStarted","Data":"90a0093549ceb4b8d569620c22a0c667eb228eac170d0be6a108904629c18f48"} Jul 07 18:54:30.964797 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:30.964774 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr"] Jul 07 18:54:30.966853 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:54:30.966822 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f7c7b22_3588_4137_ba9f_2dd8d8889ae6.slice/crio-97b6db9e95442bbd693857ca046fda0c6888cc5c013d411e495b750c1cb453a4 WatchSource:0}: Error finding container 97b6db9e95442bbd693857ca046fda0c6888cc5c013d411e495b750c1cb453a4: Status 404 returned error can't find the container with id 97b6db9e95442bbd693857ca046fda0c6888cc5c013d411e495b750c1cb453a4 Jul 07 18:54:31.915025 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:31.914949 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" event={"ID":"5f7c7b22-3588-4137-ba9f-2dd8d8889ae6","Type":"ContainerStarted","Data":"97b6db9e95442bbd693857ca046fda0c6888cc5c013d411e495b750c1cb453a4"} Jul 07 18:54:34.930901 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:34.930816 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt" event={"ID":"db38a546-4d03-4882-942c-3da5e96a7400","Type":"ContainerStarted","Data":"2bb56110772cf96710310719863aaf1dd04520b22bf9cd0821f5f4b650c27259"} Jul 07 18:54:34.931316 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:34.930914 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt" Jul 07 18:54:34.932511 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:34.932484 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" event={"ID":"5f7c7b22-3588-4137-ba9f-2dd8d8889ae6","Type":"ContainerStarted","Data":"9401e38a66404b603155b37300fd538168b1df42e132b4bbe79650f47e0cce34"} Jul 07 18:54:34.951808 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:34.951770 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt" podStartSLOduration=1.889781567 podStartE2EDuration="5.95175633s" totalImagesPullingTime="4.061974763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:54:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:54:34.950372661 +0000 UTC m=+286.920008679" watchObservedRunningTime="2026-07-07 18:54:34.95175633 +0000 UTC m=+286.921392348" Jul 07 18:54:35.938307 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:35.938220 2564 generic.go:356] "Generic (PLEG): container finished" podID="5f7c7b22-3588-4137-ba9f-2dd8d8889ae6" containerID="9401e38a66404b603155b37300fd538168b1df42e132b4bbe79650f47e0cce34" exitCode=0 Jul 07 18:54:35.938680 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:35.938303 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" event={"ID":"5f7c7b22-3588-4137-ba9f-2dd8d8889ae6","Type":"ContainerDied","Data":"9401e38a66404b603155b37300fd538168b1df42e132b4bbe79650f47e0cce34"} Jul 07 18:54:36.945089 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:36.945055 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" event={"ID":"5f7c7b22-3588-4137-ba9f-2dd8d8889ae6","Type":"ContainerStarted","Data":"a5a116cf185a1eff553f3be73b1878e9a29f31fda491cf4caf3d9afd54c78043"} Jul 07 18:54:36.945495 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:36.945158 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" Jul 07 18:54:36.971747 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:36.971697 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" podStartSLOduration=2.019543763 podStartE2EDuration="6.971684723s" totalImagesPullingTime="3.905381867s" totalInitContainerRuntime="1.046759093s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:54:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:54:36.968859824 +0000 UTC m=+288.938495843" watchObservedRunningTime="2026-07-07 18:54:36.971684723 +0000 UTC m=+288.941320739" Jul 07 18:54:45.940370 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:45.940341 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6d5b9d7dd9-kbtbt" Jul 07 18:54:47.951351 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:47.951316 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr" Jul 07 18:54:48.350478 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:48.350397 2564 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 18:54:50.405271 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:50.405236 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-85b6b9f996-bdq8n"] Jul 07 18:54:50.408701 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:50.408683 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85b6b9f996-bdq8n" Jul 07 18:54:50.411490 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:50.411462 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 18:54:50.411707 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:50.411693 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 18:54:50.411978 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:50.411960 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 18:54:50.412706 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:50.412684 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 18:54:50.412826 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:50.412739 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-nr7vx\"" Jul 07 18:54:50.426281 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:50.426260 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85b6b9f996-bdq8n"] Jul 07 18:54:50.455373 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:50.455346 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c93fe07d-9362-4713-9dee-e9217aa394e3-tmp\") pod \"kube-auth-proxy-85b6b9f996-bdq8n\" (UID: \"c93fe07d-9362-4713-9dee-e9217aa394e3\") " pod="openshift-ingress/kube-auth-proxy-85b6b9f996-bdq8n" Jul 07 18:54:50.455479 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:50.455384 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c93fe07d-9362-4713-9dee-e9217aa394e3-tls-certs\") pod \"kube-auth-proxy-85b6b9f996-bdq8n\" (UID: \"c93fe07d-9362-4713-9dee-e9217aa394e3\") " pod="openshift-ingress/kube-auth-proxy-85b6b9f996-bdq8n" Jul 07 18:54:50.455479 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:50.455404 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjpml\" (UniqueName: \"kubernetes.io/projected/c93fe07d-9362-4713-9dee-e9217aa394e3-kube-api-access-mjpml\") pod \"kube-auth-proxy-85b6b9f996-bdq8n\" (UID: \"c93fe07d-9362-4713-9dee-e9217aa394e3\") " pod="openshift-ingress/kube-auth-proxy-85b6b9f996-bdq8n" Jul 07 18:54:50.556132 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:50.556104 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c93fe07d-9362-4713-9dee-e9217aa394e3-tmp\") pod \"kube-auth-proxy-85b6b9f996-bdq8n\" (UID: \"c93fe07d-9362-4713-9dee-e9217aa394e3\") " pod="openshift-ingress/kube-auth-proxy-85b6b9f996-bdq8n" Jul 07 18:54:50.556329 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:50.556142 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c93fe07d-9362-4713-9dee-e9217aa394e3-tls-certs\") pod \"kube-auth-proxy-85b6b9f996-bdq8n\" (UID: \"c93fe07d-9362-4713-9dee-e9217aa394e3\") " pod="openshift-ingress/kube-auth-proxy-85b6b9f996-bdq8n" Jul 07 18:54:50.556329 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:50.556160 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjpml\" (UniqueName: \"kubernetes.io/projected/c93fe07d-9362-4713-9dee-e9217aa394e3-kube-api-access-mjpml\") pod \"kube-auth-proxy-85b6b9f996-bdq8n\" (UID: \"c93fe07d-9362-4713-9dee-e9217aa394e3\") " pod="openshift-ingress/kube-auth-proxy-85b6b9f996-bdq8n" Jul 07 18:54:50.556329 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:54:50.556290 2564 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 07 18:54:50.556505 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:54:50.556372 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c93fe07d-9362-4713-9dee-e9217aa394e3-tls-certs podName:c93fe07d-9362-4713-9dee-e9217aa394e3 nodeName:}" failed. No retries permitted until 2026-07-07 18:54:51.056350861 +0000 UTC m=+303.025986857 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/c93fe07d-9362-4713-9dee-e9217aa394e3-tls-certs") pod "kube-auth-proxy-85b6b9f996-bdq8n" (UID: "c93fe07d-9362-4713-9dee-e9217aa394e3") : secret "kube-auth-proxy-tls" not found Jul 07 18:54:50.558448 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:50.558426 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c93fe07d-9362-4713-9dee-e9217aa394e3-tmp\") pod \"kube-auth-proxy-85b6b9f996-bdq8n\" (UID: \"c93fe07d-9362-4713-9dee-e9217aa394e3\") " pod="openshift-ingress/kube-auth-proxy-85b6b9f996-bdq8n" Jul 07 18:54:50.568497 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:50.568475 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjpml\" (UniqueName: \"kubernetes.io/projected/c93fe07d-9362-4713-9dee-e9217aa394e3-kube-api-access-mjpml\") pod \"kube-auth-proxy-85b6b9f996-bdq8n\" (UID: \"c93fe07d-9362-4713-9dee-e9217aa394e3\") " pod="openshift-ingress/kube-auth-proxy-85b6b9f996-bdq8n" Jul 07 18:54:51.058580 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:51.058549 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c93fe07d-9362-4713-9dee-e9217aa394e3-tls-certs\") pod \"kube-auth-proxy-85b6b9f996-bdq8n\" (UID: \"c93fe07d-9362-4713-9dee-e9217aa394e3\") " pod="openshift-ingress/kube-auth-proxy-85b6b9f996-bdq8n" Jul 07 18:54:51.060902 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:51.060874 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c93fe07d-9362-4713-9dee-e9217aa394e3-tls-certs\") pod \"kube-auth-proxy-85b6b9f996-bdq8n\" (UID: \"c93fe07d-9362-4713-9dee-e9217aa394e3\") " pod="openshift-ingress/kube-auth-proxy-85b6b9f996-bdq8n" Jul 07 18:54:51.318717 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:51.318642 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85b6b9f996-bdq8n" Jul 07 18:54:51.458656 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:51.458633 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85b6b9f996-bdq8n"] Jul 07 18:54:51.461083 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:54:51.461054 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc93fe07d_9362_4713_9dee_e9217aa394e3.slice/crio-62ee4c340df942c554bf99ed32f905395bece6caa201f20912cd0805a42c6fc1 WatchSource:0}: Error finding container 62ee4c340df942c554bf99ed32f905395bece6caa201f20912cd0805a42c6fc1: Status 404 returned error can't find the container with id 62ee4c340df942c554bf99ed32f905395bece6caa201f20912cd0805a42c6fc1 Jul 07 18:54:51.463043 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:51.463027 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 18:54:52.008012 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:52.007978 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85b6b9f996-bdq8n" event={"ID":"c93fe07d-9362-4713-9dee-e9217aa394e3","Type":"ContainerStarted","Data":"62ee4c340df942c554bf99ed32f905395bece6caa201f20912cd0805a42c6fc1"} Jul 07 18:54:55.022973 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:55.022932 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85b6b9f996-bdq8n" event={"ID":"c93fe07d-9362-4713-9dee-e9217aa394e3","Type":"ContainerStarted","Data":"e9366e11e1811cbfd4b5eadf19e1c0b8235afd328f3a7aeb4b90f39cda9fdfd3"} Jul 07 18:54:55.042315 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:54:55.042270 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-85b6b9f996-bdq8n" podStartSLOduration=2.13915622 podStartE2EDuration="5.042254558s" totalImagesPullingTime="2.903098338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:54:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:54:55.040920775 +0000 UTC m=+307.010556784" watchObservedRunningTime="2026-07-07 18:54:55.042254558 +0000 UTC m=+307.011890574" Jul 07 18:56:36.878457 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:36.878423 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9b4bdcc46-7nw9b"] Jul 07 18:56:36.881777 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:36.881759 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:36.885104 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:36.885083 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 18:56:36.885308 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:36.885293 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zh69j\"" Jul 07 18:56:36.886059 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:36.886041 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 18:56:36.886182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:36.886123 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 18:56:36.886182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:36.886133 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 18:56:36.886182 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:36.886154 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 18:56:36.886406 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:36.886188 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 18:56:36.886520 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:36.886507 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 18:56:36.896339 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:36.896318 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 18:56:36.901473 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:36.901450 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9b4bdcc46-7nw9b"] Jul 07 18:56:36.961143 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:36.961117 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ec7a05c-4b58-435b-bf38-c5134e09fae7-console-config\") pod \"console-9b4bdcc46-7nw9b\" (UID: \"1ec7a05c-4b58-435b-bf38-c5134e09fae7\") " pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:36.961283 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:36.961149 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ec7a05c-4b58-435b-bf38-c5134e09fae7-service-ca\") pod \"console-9b4bdcc46-7nw9b\" (UID: \"1ec7a05c-4b58-435b-bf38-c5134e09fae7\") " pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:36.961283 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:36.961182 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ec7a05c-4b58-435b-bf38-c5134e09fae7-console-serving-cert\") pod \"console-9b4bdcc46-7nw9b\" (UID: \"1ec7a05c-4b58-435b-bf38-c5134e09fae7\") " pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:36.961283 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:36.961240 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ec7a05c-4b58-435b-bf38-c5134e09fae7-console-oauth-config\") pod \"console-9b4bdcc46-7nw9b\" (UID: \"1ec7a05c-4b58-435b-bf38-c5134e09fae7\") " pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:36.961283 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:36.961273 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ec7a05c-4b58-435b-bf38-c5134e09fae7-trusted-ca-bundle\") pod \"console-9b4bdcc46-7nw9b\" (UID: \"1ec7a05c-4b58-435b-bf38-c5134e09fae7\") " pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:36.961478 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:36.961333 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ec7a05c-4b58-435b-bf38-c5134e09fae7-oauth-serving-cert\") pod \"console-9b4bdcc46-7nw9b\" (UID: \"1ec7a05c-4b58-435b-bf38-c5134e09fae7\") " pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:36.961478 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:36.961378 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57ls6\" (UniqueName: \"kubernetes.io/projected/1ec7a05c-4b58-435b-bf38-c5134e09fae7-kube-api-access-57ls6\") pod \"console-9b4bdcc46-7nw9b\" (UID: \"1ec7a05c-4b58-435b-bf38-c5134e09fae7\") " pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:37.062484 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:37.062463 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ec7a05c-4b58-435b-bf38-c5134e09fae7-console-serving-cert\") pod \"console-9b4bdcc46-7nw9b\" (UID: \"1ec7a05c-4b58-435b-bf38-c5134e09fae7\") " pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:37.062595 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:37.062494 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ec7a05c-4b58-435b-bf38-c5134e09fae7-console-oauth-config\") pod \"console-9b4bdcc46-7nw9b\" (UID: \"1ec7a05c-4b58-435b-bf38-c5134e09fae7\") " pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:37.062595 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:37.062527 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ec7a05c-4b58-435b-bf38-c5134e09fae7-trusted-ca-bundle\") pod \"console-9b4bdcc46-7nw9b\" (UID: \"1ec7a05c-4b58-435b-bf38-c5134e09fae7\") " pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:37.062595 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:37.062567 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ec7a05c-4b58-435b-bf38-c5134e09fae7-oauth-serving-cert\") pod \"console-9b4bdcc46-7nw9b\" (UID: \"1ec7a05c-4b58-435b-bf38-c5134e09fae7\") " pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:37.062755 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:37.062594 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57ls6\" (UniqueName: \"kubernetes.io/projected/1ec7a05c-4b58-435b-bf38-c5134e09fae7-kube-api-access-57ls6\") pod \"console-9b4bdcc46-7nw9b\" (UID: \"1ec7a05c-4b58-435b-bf38-c5134e09fae7\") " pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:37.062755 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:37.062665 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ec7a05c-4b58-435b-bf38-c5134e09fae7-console-config\") pod \"console-9b4bdcc46-7nw9b\" (UID: \"1ec7a05c-4b58-435b-bf38-c5134e09fae7\") " pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:37.062755 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:37.062691 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ec7a05c-4b58-435b-bf38-c5134e09fae7-service-ca\") pod \"console-9b4bdcc46-7nw9b\" (UID: \"1ec7a05c-4b58-435b-bf38-c5134e09fae7\") " pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:37.063404 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:37.063380 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1ec7a05c-4b58-435b-bf38-c5134e09fae7-oauth-serving-cert\") pod \"console-9b4bdcc46-7nw9b\" (UID: \"1ec7a05c-4b58-435b-bf38-c5134e09fae7\") " pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:37.063515 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:37.063432 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1ec7a05c-4b58-435b-bf38-c5134e09fae7-trusted-ca-bundle\") pod \"console-9b4bdcc46-7nw9b\" (UID: \"1ec7a05c-4b58-435b-bf38-c5134e09fae7\") " pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:37.063515 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:37.063443 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1ec7a05c-4b58-435b-bf38-c5134e09fae7-console-config\") pod \"console-9b4bdcc46-7nw9b\" (UID: \"1ec7a05c-4b58-435b-bf38-c5134e09fae7\") " pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:37.063515 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:37.063504 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1ec7a05c-4b58-435b-bf38-c5134e09fae7-service-ca\") pod \"console-9b4bdcc46-7nw9b\" (UID: \"1ec7a05c-4b58-435b-bf38-c5134e09fae7\") " pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:37.064893 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:37.064868 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1ec7a05c-4b58-435b-bf38-c5134e09fae7-console-oauth-config\") pod \"console-9b4bdcc46-7nw9b\" (UID: \"1ec7a05c-4b58-435b-bf38-c5134e09fae7\") " pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:37.064990 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:37.064978 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1ec7a05c-4b58-435b-bf38-c5134e09fae7-console-serving-cert\") pod \"console-9b4bdcc46-7nw9b\" (UID: \"1ec7a05c-4b58-435b-bf38-c5134e09fae7\") " pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:37.074221 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:37.074187 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57ls6\" (UniqueName: \"kubernetes.io/projected/1ec7a05c-4b58-435b-bf38-c5134e09fae7-kube-api-access-57ls6\") pod \"console-9b4bdcc46-7nw9b\" (UID: \"1ec7a05c-4b58-435b-bf38-c5134e09fae7\") " pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:37.190517 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:37.190490 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:37.330912 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:37.330833 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9b4bdcc46-7nw9b"] Jul 07 18:56:37.333470 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:56:37.333443 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ec7a05c_4b58_435b_bf38_c5134e09fae7.slice/crio-32fede45b50274e4bb8c6aa36e2e2f14a7f55919841c78fb0766ddae0dd12347 WatchSource:0}: Error finding container 32fede45b50274e4bb8c6aa36e2e2f14a7f55919841c78fb0766ddae0dd12347: Status 404 returned error can't find the container with id 32fede45b50274e4bb8c6aa36e2e2f14a7f55919841c78fb0766ddae0dd12347 Jul 07 18:56:37.458638 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:37.458569 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9b4bdcc46-7nw9b" event={"ID":"1ec7a05c-4b58-435b-bf38-c5134e09fae7","Type":"ContainerStarted","Data":"4307b7c2544bde30948605b60f8cc9b4f363d85a42b5553bd2d31c2ecb5b31e5"} Jul 07 18:56:37.458638 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:37.458615 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9b4bdcc46-7nw9b" event={"ID":"1ec7a05c-4b58-435b-bf38-c5134e09fae7","Type":"ContainerStarted","Data":"32fede45b50274e4bb8c6aa36e2e2f14a7f55919841c78fb0766ddae0dd12347"} Jul 07 18:56:37.483913 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:37.483869 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9b4bdcc46-7nw9b" podStartSLOduration=1.483852183 podStartE2EDuration="1.483852183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:56:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:56:37.482517771 +0000 UTC m=+409.452153777" watchObservedRunningTime="2026-07-07 18:56:37.483852183 +0000 UTC m=+409.453488198" Jul 07 18:56:47.190640 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:47.190596 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:47.191012 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:47.190659 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:47.195296 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:47.195264 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:47.505482 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:47.505401 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9b4bdcc46-7nw9b" Jul 07 18:56:53.972384 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:53.972350 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-gdzvc"] Jul 07 18:56:53.975671 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:53.975654 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-gdzvc" Jul 07 18:56:53.978352 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:56:53.978315 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-135-81.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-81.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap" Jul 07 18:56:53.978352 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:56:53.978314 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"kuadrant-operator-controller-manager-dockercfg-hpcqw\" is forbidden: User \"system:node:ip-10-0-135-81.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-81.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hpcqw\"" type="*v1.Secret" Jul 07 18:56:53.978352 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:56:53.978334 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-gdzvc\" is forbidden: User \"system:node:ip-10-0-135-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-81.ec2.internal' and this object" podUID="1924fdd9-68fb-4130-b678-05650c95e2b3" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-gdzvc" Jul 07 18:56:53.978352 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:56:53.978329 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-135-81.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-81.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap" Jul 07 18:56:53.996521 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:53.996488 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-gdzvc"] Jul 07 18:56:54.095774 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.095751 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pckm\" (UniqueName: \"kubernetes.io/projected/1924fdd9-68fb-4130-b678-05650c95e2b3-kube-api-access-8pckm\") pod \"kuadrant-operator-controller-manager-55467d9f7-gdzvc\" (UID: \"1924fdd9-68fb-4130-b678-05650c95e2b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-gdzvc" Jul 07 18:56:54.095888 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.095805 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1924fdd9-68fb-4130-b678-05650c95e2b3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-gdzvc\" (UID: \"1924fdd9-68fb-4130-b678-05650c95e2b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-gdzvc" Jul 07 18:56:54.196522 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.196494 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pckm\" (UniqueName: \"kubernetes.io/projected/1924fdd9-68fb-4130-b678-05650c95e2b3-kube-api-access-8pckm\") pod \"kuadrant-operator-controller-manager-55467d9f7-gdzvc\" (UID: \"1924fdd9-68fb-4130-b678-05650c95e2b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-gdzvc" Jul 07 18:56:54.196634 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.196565 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1924fdd9-68fb-4130-b678-05650c95e2b3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-gdzvc\" (UID: \"1924fdd9-68fb-4130-b678-05650c95e2b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-gdzvc" Jul 07 18:56:54.196879 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.196862 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1924fdd9-68fb-4130-b678-05650c95e2b3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-gdzvc\" (UID: \"1924fdd9-68fb-4130-b678-05650c95e2b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-gdzvc" Jul 07 18:56:54.215257 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.215231 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-gdzvc"] Jul 07 18:56:54.215440 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:56:54.215423 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-8pckm], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-gdzvc" podUID="1924fdd9-68fb-4130-b678-05650c95e2b3" Jul 07 18:56:54.221430 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.221411 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-gdzvc"] Jul 07 18:56:54.249601 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.249542 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6"] Jul 07 18:56:54.252948 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.252931 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6" Jul 07 18:56:54.278127 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.278106 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6"] Jul 07 18:56:54.310397 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:56:54.310370 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-gdzvc\" is forbidden: User \"system:node:ip-10-0-135-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-81.ec2.internal' and this object" podUID="1924fdd9-68fb-4130-b678-05650c95e2b3" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-gdzvc" Jul 07 18:56:54.397990 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.397966 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0a63297-8dbf-46d0-9d83-7f9559178d1b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-drpp6\" (UID: \"a0a63297-8dbf-46d0-9d83-7f9559178d1b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6" Jul 07 18:56:54.398105 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.398019 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4zm7\" (UniqueName: \"kubernetes.io/projected/a0a63297-8dbf-46d0-9d83-7f9559178d1b-kube-api-access-w4zm7\") pod \"kuadrant-operator-controller-manager-55467d9f7-drpp6\" (UID: \"a0a63297-8dbf-46d0-9d83-7f9559178d1b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6" Jul 07 18:56:54.499444 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.499418 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0a63297-8dbf-46d0-9d83-7f9559178d1b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-drpp6\" (UID: \"a0a63297-8dbf-46d0-9d83-7f9559178d1b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6" Jul 07 18:56:54.499526 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.499467 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4zm7\" (UniqueName: \"kubernetes.io/projected/a0a63297-8dbf-46d0-9d83-7f9559178d1b-kube-api-access-w4zm7\") pod \"kuadrant-operator-controller-manager-55467d9f7-drpp6\" (UID: \"a0a63297-8dbf-46d0-9d83-7f9559178d1b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6" Jul 07 18:56:54.499797 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.499757 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0a63297-8dbf-46d0-9d83-7f9559178d1b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-drpp6\" (UID: \"a0a63297-8dbf-46d0-9d83-7f9559178d1b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6" Jul 07 18:56:54.535291 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.535265 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-gdzvc" Jul 07 18:56:54.538909 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:56:54.538876 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-gdzvc\" is forbidden: User \"system:node:ip-10-0-135-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-81.ec2.internal' and this object" podUID="1924fdd9-68fb-4130-b678-05650c95e2b3" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-gdzvc" Jul 07 18:56:54.541522 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.541506 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-gdzvc" Jul 07 18:56:54.543806 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:56:54.543789 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-gdzvc\" is forbidden: User \"system:node:ip-10-0-135-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-81.ec2.internal' and this object" podUID="1924fdd9-68fb-4130-b678-05650c95e2b3" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-gdzvc" Jul 07 18:56:54.600297 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.600263 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1924fdd9-68fb-4130-b678-05650c95e2b3-extensions-socket-volume\") pod \"1924fdd9-68fb-4130-b678-05650c95e2b3\" (UID: \"1924fdd9-68fb-4130-b678-05650c95e2b3\") " Jul 07 18:56:54.600497 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.600477 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1924fdd9-68fb-4130-b678-05650c95e2b3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "1924fdd9-68fb-4130-b678-05650c95e2b3" (UID: "1924fdd9-68fb-4130-b678-05650c95e2b3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:56:54.701290 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.701264 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/1924fdd9-68fb-4130-b678-05650c95e2b3-extensions-socket-volume\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:56:54.806251 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.806173 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 18:56:54.906189 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.906166 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 18:56:54.909468 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.909448 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4zm7\" (UniqueName: \"kubernetes.io/projected/a0a63297-8dbf-46d0-9d83-7f9559178d1b-kube-api-access-w4zm7\") pod \"kuadrant-operator-controller-manager-55467d9f7-drpp6\" (UID: \"a0a63297-8dbf-46d0-9d83-7f9559178d1b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6" Jul 07 18:56:54.910972 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:54.910955 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pckm\" (UniqueName: \"kubernetes.io/projected/1924fdd9-68fb-4130-b678-05650c95e2b3-kube-api-access-8pckm\") pod \"kuadrant-operator-controller-manager-55467d9f7-gdzvc\" (UID: \"1924fdd9-68fb-4130-b678-05650c95e2b3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-gdzvc" Jul 07 18:56:55.002998 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:55.002964 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pckm\" (UniqueName: \"kubernetes.io/projected/1924fdd9-68fb-4130-b678-05650c95e2b3-kube-api-access-8pckm\") pod \"1924fdd9-68fb-4130-b678-05650c95e2b3\" (UID: \"1924fdd9-68fb-4130-b678-05650c95e2b3\") " Jul 07 18:56:55.004716 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:55.004694 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1924fdd9-68fb-4130-b678-05650c95e2b3-kube-api-access-8pckm" (OuterVolumeSpecName: "kube-api-access-8pckm") pod "1924fdd9-68fb-4130-b678-05650c95e2b3" (UID: "1924fdd9-68fb-4130-b678-05650c95e2b3"). InnerVolumeSpecName "kube-api-access-8pckm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:56:55.103710 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:55.103643 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pckm\" (UniqueName: \"kubernetes.io/projected/1924fdd9-68fb-4130-b678-05650c95e2b3-kube-api-access-8pckm\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:56:55.338312 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:55.338288 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hpcqw\"" Jul 07 18:56:55.345339 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:55.345323 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6" Jul 07 18:56:55.480386 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:55.480360 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6"] Jul 07 18:56:55.483154 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:56:55.483131 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0a63297_8dbf_46d0_9d83_7f9559178d1b.slice/crio-a0fdcfd2c250cb611789c9046cbb96887adf8a9bd2b2800cf073e91d738807a9 WatchSource:0}: Error finding container a0fdcfd2c250cb611789c9046cbb96887adf8a9bd2b2800cf073e91d738807a9: Status 404 returned error can't find the container with id a0fdcfd2c250cb611789c9046cbb96887adf8a9bd2b2800cf073e91d738807a9 Jul 07 18:56:55.540535 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:55.540508 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-gdzvc" Jul 07 18:56:55.540645 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:55.540512 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6" event={"ID":"a0a63297-8dbf-46d0-9d83-7f9559178d1b","Type":"ContainerStarted","Data":"a0fdcfd2c250cb611789c9046cbb96887adf8a9bd2b2800cf073e91d738807a9"} Jul 07 18:56:55.544157 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:56:55.544133 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-gdzvc\" is forbidden: User \"system:node:ip-10-0-135-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-81.ec2.internal' and this object" podUID="1924fdd9-68fb-4130-b678-05650c95e2b3" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-gdzvc" Jul 07 18:56:55.549498 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:56:55.549475 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-gdzvc\" is forbidden: User \"system:node:ip-10-0-135-81.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-81.ec2.internal' and this object" podUID="1924fdd9-68fb-4130-b678-05650c95e2b3" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-gdzvc" Jul 07 18:56:56.578160 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:56:56.578126 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1924fdd9-68fb-4130-b678-05650c95e2b3" path="/var/lib/kubelet/pods/1924fdd9-68fb-4130-b678-05650c95e2b3/volumes" Jul 07 18:57:01.568296 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:01.568256 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6" event={"ID":"a0a63297-8dbf-46d0-9d83-7f9559178d1b","Type":"ContainerStarted","Data":"990be047097f19f917c6dfafffccc7ba85888d771051443067ec84595283fce5"} Jul 07 18:57:01.568706 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:01.568345 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6" Jul 07 18:57:01.596855 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:01.596813 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6" podStartSLOduration=2.349359088 podStartE2EDuration="7.596798419s" totalImagesPullingTime="5.247439331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:56:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:57:01.595828129 +0000 UTC m=+433.565464163" watchObservedRunningTime="2026-07-07 18:57:01.596798419 +0000 UTC m=+433.566434500" Jul 07 18:57:12.574669 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:12.574591 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6" Jul 07 18:57:24.624125 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:24.624094 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6"] Jul 07 18:57:24.624586 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:24.624302 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6" podUID="a0a63297-8dbf-46d0-9d83-7f9559178d1b" containerName="manager" containerID="cri-o://990be047097f19f917c6dfafffccc7ba85888d771051443067ec84595283fce5" gracePeriod=10 Jul 07 18:57:24.868825 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:24.868802 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6" Jul 07 18:57:24.934380 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:24.934304 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0a63297-8dbf-46d0-9d83-7f9559178d1b-extensions-socket-volume\") pod \"a0a63297-8dbf-46d0-9d83-7f9559178d1b\" (UID: \"a0a63297-8dbf-46d0-9d83-7f9559178d1b\") " Jul 07 18:57:24.934380 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:24.934359 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4zm7\" (UniqueName: \"kubernetes.io/projected/a0a63297-8dbf-46d0-9d83-7f9559178d1b-kube-api-access-w4zm7\") pod \"a0a63297-8dbf-46d0-9d83-7f9559178d1b\" (UID: \"a0a63297-8dbf-46d0-9d83-7f9559178d1b\") " Jul 07 18:57:24.934759 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:24.934735 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0a63297-8dbf-46d0-9d83-7f9559178d1b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a0a63297-8dbf-46d0-9d83-7f9559178d1b" (UID: "a0a63297-8dbf-46d0-9d83-7f9559178d1b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 18:57:24.936414 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:24.936389 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0a63297-8dbf-46d0-9d83-7f9559178d1b-kube-api-access-w4zm7" (OuterVolumeSpecName: "kube-api-access-w4zm7") pod "a0a63297-8dbf-46d0-9d83-7f9559178d1b" (UID: "a0a63297-8dbf-46d0-9d83-7f9559178d1b"). InnerVolumeSpecName "kube-api-access-w4zm7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:57:25.035814 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:25.035781 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4zm7\" (UniqueName: \"kubernetes.io/projected/a0a63297-8dbf-46d0-9d83-7f9559178d1b-kube-api-access-w4zm7\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:57:25.035814 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:25.035808 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0a63297-8dbf-46d0-9d83-7f9559178d1b-extensions-socket-volume\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:57:25.679831 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:25.679796 2564 generic.go:356] "Generic (PLEG): container finished" podID="a0a63297-8dbf-46d0-9d83-7f9559178d1b" containerID="990be047097f19f917c6dfafffccc7ba85888d771051443067ec84595283fce5" exitCode=0 Jul 07 18:57:25.679831 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:25.679835 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6" Jul 07 18:57:25.680283 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:25.679874 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6" event={"ID":"a0a63297-8dbf-46d0-9d83-7f9559178d1b","Type":"ContainerDied","Data":"990be047097f19f917c6dfafffccc7ba85888d771051443067ec84595283fce5"} Jul 07 18:57:25.680283 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:25.679925 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6" event={"ID":"a0a63297-8dbf-46d0-9d83-7f9559178d1b","Type":"ContainerDied","Data":"a0fdcfd2c250cb611789c9046cbb96887adf8a9bd2b2800cf073e91d738807a9"} Jul 07 18:57:25.680283 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:25.679946 2564 scope.go:117] "RemoveContainer" containerID="990be047097f19f917c6dfafffccc7ba85888d771051443067ec84595283fce5" Jul 07 18:57:25.690586 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:25.690538 2564 scope.go:117] "RemoveContainer" containerID="990be047097f19f917c6dfafffccc7ba85888d771051443067ec84595283fce5" Jul 07 18:57:25.691017 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:57:25.690992 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"990be047097f19f917c6dfafffccc7ba85888d771051443067ec84595283fce5\": container with ID starting with 990be047097f19f917c6dfafffccc7ba85888d771051443067ec84595283fce5 not found: ID does not exist" containerID="990be047097f19f917c6dfafffccc7ba85888d771051443067ec84595283fce5" Jul 07 18:57:25.691102 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:25.691030 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"990be047097f19f917c6dfafffccc7ba85888d771051443067ec84595283fce5"} err="failed to get container status \"990be047097f19f917c6dfafffccc7ba85888d771051443067ec84595283fce5\": rpc error: code = NotFound desc = could not find container \"990be047097f19f917c6dfafffccc7ba85888d771051443067ec84595283fce5\": container with ID starting with 990be047097f19f917c6dfafffccc7ba85888d771051443067ec84595283fce5 not found: ID does not exist" Jul 07 18:57:25.706941 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:25.706921 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6"] Jul 07 18:57:25.715040 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:25.715015 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-drpp6"] Jul 07 18:57:26.574232 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:26.574180 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0a63297-8dbf-46d0-9d83-7f9559178d1b" path="/var/lib/kubelet/pods/a0a63297-8dbf-46d0-9d83-7f9559178d1b/volumes" Jul 07 18:57:44.715031 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:44.714999 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s8qcl"] Jul 07 18:57:44.715555 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:44.715353 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0a63297-8dbf-46d0-9d83-7f9559178d1b" containerName="manager" Jul 07 18:57:44.715555 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:44.715365 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0a63297-8dbf-46d0-9d83-7f9559178d1b" containerName="manager" Jul 07 18:57:44.715555 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:44.715433 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0a63297-8dbf-46d0-9d83-7f9559178d1b" containerName="manager" Jul 07 18:57:44.718293 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:44.718275 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-s8qcl" Jul 07 18:57:44.720966 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:44.720939 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 18:57:44.721084 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:44.721037 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 18:57:44.721658 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:44.721641 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 18:57:44.721721 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:44.721644 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gpj4b\"" Jul 07 18:57:44.727372 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:44.727351 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s8qcl"] Jul 07 18:57:44.804139 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:44.804108 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2a7a9358-2829-403a-b7af-10ffb34e7ea5-config-file\") pod \"limitador-limitador-64c94fd64f-s8qcl\" (UID: \"2a7a9358-2829-403a-b7af-10ffb34e7ea5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s8qcl" Jul 07 18:57:44.804331 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:44.804189 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6vkz\" (UniqueName: \"kubernetes.io/projected/2a7a9358-2829-403a-b7af-10ffb34e7ea5-kube-api-access-c6vkz\") pod \"limitador-limitador-64c94fd64f-s8qcl\" (UID: \"2a7a9358-2829-403a-b7af-10ffb34e7ea5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s8qcl" Jul 07 18:57:44.811667 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:44.811635 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s8qcl"] Jul 07 18:57:44.905567 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:44.905532 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2a7a9358-2829-403a-b7af-10ffb34e7ea5-config-file\") pod \"limitador-limitador-64c94fd64f-s8qcl\" (UID: \"2a7a9358-2829-403a-b7af-10ffb34e7ea5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s8qcl" Jul 07 18:57:44.905741 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:44.905590 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6vkz\" (UniqueName: \"kubernetes.io/projected/2a7a9358-2829-403a-b7af-10ffb34e7ea5-kube-api-access-c6vkz\") pod \"limitador-limitador-64c94fd64f-s8qcl\" (UID: \"2a7a9358-2829-403a-b7af-10ffb34e7ea5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s8qcl" Jul 07 18:57:44.906171 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:44.906149 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2a7a9358-2829-403a-b7af-10ffb34e7ea5-config-file\") pod \"limitador-limitador-64c94fd64f-s8qcl\" (UID: \"2a7a9358-2829-403a-b7af-10ffb34e7ea5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s8qcl" Jul 07 18:57:44.918258 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:44.918231 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6vkz\" (UniqueName: \"kubernetes.io/projected/2a7a9358-2829-403a-b7af-10ffb34e7ea5-kube-api-access-c6vkz\") pod \"limitador-limitador-64c94fd64f-s8qcl\" (UID: \"2a7a9358-2829-403a-b7af-10ffb34e7ea5\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-s8qcl" Jul 07 18:57:45.028487 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:45.028393 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-s8qcl" Jul 07 18:57:45.155640 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:45.155610 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s8qcl"] Jul 07 18:57:45.158757 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:57:45.158728 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a7a9358_2829_403a_b7af_10ffb34e7ea5.slice/crio-fba884674c646e223cdb3c4850ccb3aa40ddee48bfc4b308123aae650e95381e WatchSource:0}: Error finding container fba884674c646e223cdb3c4850ccb3aa40ddee48bfc4b308123aae650e95381e: Status 404 returned error can't find the container with id fba884674c646e223cdb3c4850ccb3aa40ddee48bfc4b308123aae650e95381e Jul 07 18:57:45.596695 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:45.596662 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-8njz4"] Jul 07 18:57:45.601083 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:45.601065 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8njz4" Jul 07 18:57:45.603417 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:45.603394 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-9spzj\"" Jul 07 18:57:45.611448 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:45.611419 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-8njz4"] Jul 07 18:57:45.712997 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:45.712959 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tklrl\" (UniqueName: \"kubernetes.io/projected/e1dd952b-fb36-4374-afc5-1dfae52bc60d-kube-api-access-tklrl\") pod \"authorino-58cd955675-8njz4\" (UID: \"e1dd952b-fb36-4374-afc5-1dfae52bc60d\") " pod="kuadrant-system/authorino-58cd955675-8njz4" Jul 07 18:57:45.771216 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:45.771124 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-s8qcl" event={"ID":"2a7a9358-2829-403a-b7af-10ffb34e7ea5","Type":"ContainerStarted","Data":"fba884674c646e223cdb3c4850ccb3aa40ddee48bfc4b308123aae650e95381e"} Jul 07 18:57:45.820029 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:45.815085 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tklrl\" (UniqueName: \"kubernetes.io/projected/e1dd952b-fb36-4374-afc5-1dfae52bc60d-kube-api-access-tklrl\") pod \"authorino-58cd955675-8njz4\" (UID: \"e1dd952b-fb36-4374-afc5-1dfae52bc60d\") " pod="kuadrant-system/authorino-58cd955675-8njz4" Jul 07 18:57:45.820029 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:45.818286 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-m2wd4"] Jul 07 18:57:45.822171 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:45.822129 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-m2wd4" Jul 07 18:57:45.847437 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:45.847329 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tklrl\" (UniqueName: \"kubernetes.io/projected/e1dd952b-fb36-4374-afc5-1dfae52bc60d-kube-api-access-tklrl\") pod \"authorino-58cd955675-8njz4\" (UID: \"e1dd952b-fb36-4374-afc5-1dfae52bc60d\") " pod="kuadrant-system/authorino-58cd955675-8njz4" Jul 07 18:57:45.851601 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:45.851581 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m2wd4"] Jul 07 18:57:45.911086 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:45.911038 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8njz4" Jul 07 18:57:45.916788 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:45.916648 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq94r\" (UniqueName: \"kubernetes.io/projected/281dcb4a-48c1-4720-9059-4d03d7e4eea1-kube-api-access-nq94r\") pod \"authorino-75d556bd77-m2wd4\" (UID: \"281dcb4a-48c1-4720-9059-4d03d7e4eea1\") " pod="kuadrant-system/authorino-75d556bd77-m2wd4" Jul 07 18:57:46.020902 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:46.020506 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq94r\" (UniqueName: \"kubernetes.io/projected/281dcb4a-48c1-4720-9059-4d03d7e4eea1-kube-api-access-nq94r\") pod \"authorino-75d556bd77-m2wd4\" (UID: \"281dcb4a-48c1-4720-9059-4d03d7e4eea1\") " pod="kuadrant-system/authorino-75d556bd77-m2wd4" Jul 07 18:57:46.030949 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:46.030899 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq94r\" (UniqueName: \"kubernetes.io/projected/281dcb4a-48c1-4720-9059-4d03d7e4eea1-kube-api-access-nq94r\") pod \"authorino-75d556bd77-m2wd4\" (UID: \"281dcb4a-48c1-4720-9059-4d03d7e4eea1\") " pod="kuadrant-system/authorino-75d556bd77-m2wd4" Jul 07 18:57:46.077633 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:46.077598 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-8njz4"] Jul 07 18:57:46.080879 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:57:46.080847 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1dd952b_fb36_4374_afc5_1dfae52bc60d.slice/crio-6676adb9db0f6b8af79b7d80de79203bada32b016acbb225d7233a4ef6cab901 WatchSource:0}: Error finding container 6676adb9db0f6b8af79b7d80de79203bada32b016acbb225d7233a4ef6cab901: Status 404 returned error can't find the container with id 6676adb9db0f6b8af79b7d80de79203bada32b016acbb225d7233a4ef6cab901 Jul 07 18:57:46.156138 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:46.156069 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-m2wd4" Jul 07 18:57:46.310831 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:46.310801 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m2wd4"] Jul 07 18:57:46.312702 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:57:46.312675 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod281dcb4a_48c1_4720_9059_4d03d7e4eea1.slice/crio-33cc5a6ff1cfc5213fce58bdae30edc449d44aa40644b7393cdc2d2f6c1cd416 WatchSource:0}: Error finding container 33cc5a6ff1cfc5213fce58bdae30edc449d44aa40644b7393cdc2d2f6c1cd416: Status 404 returned error can't find the container with id 33cc5a6ff1cfc5213fce58bdae30edc449d44aa40644b7393cdc2d2f6c1cd416 Jul 07 18:57:46.777180 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:46.777142 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-m2wd4" event={"ID":"281dcb4a-48c1-4720-9059-4d03d7e4eea1","Type":"ContainerStarted","Data":"33cc5a6ff1cfc5213fce58bdae30edc449d44aa40644b7393cdc2d2f6c1cd416"} Jul 07 18:57:46.778545 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:46.778510 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8njz4" event={"ID":"e1dd952b-fb36-4374-afc5-1dfae52bc60d","Type":"ContainerStarted","Data":"6676adb9db0f6b8af79b7d80de79203bada32b016acbb225d7233a4ef6cab901"} Jul 07 18:57:48.795053 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:48.795007 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-s8qcl" event={"ID":"2a7a9358-2829-403a-b7af-10ffb34e7ea5","Type":"ContainerStarted","Data":"8a9e667e3776190407e08bd35793c643c6737a12a5ce576adffdccebb62d5df3"} Jul 07 18:57:48.795554 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:48.795095 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-s8qcl" Jul 07 18:57:48.815797 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:48.815736 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-s8qcl" podStartSLOduration=1.672731778 podStartE2EDuration="4.815717977s" totalImagesPullingTime="3.142986199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:57:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:57:48.814506934 +0000 UTC m=+480.784142949" watchObservedRunningTime="2026-07-07 18:57:48.815717977 +0000 UTC m=+480.785353995" Jul 07 18:57:50.805799 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:50.805759 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-m2wd4" event={"ID":"281dcb4a-48c1-4720-9059-4d03d7e4eea1","Type":"ContainerStarted","Data":"8484b06c164f58b4c60dcbeff681e9b8743a341f982c5fa48c14739612df23e4"} Jul 07 18:57:50.807219 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:50.807175 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8njz4" event={"ID":"e1dd952b-fb36-4374-afc5-1dfae52bc60d","Type":"ContainerStarted","Data":"246818cf3e851d93f8a3687c47c83f8713f036cf93c989de1037db0f501f09ee"} Jul 07 18:57:50.824516 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:50.824464 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-m2wd4" podStartSLOduration=1.8166352909999999 podStartE2EDuration="5.824448106s" totalImagesPullingTime="4.007812815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:57:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:57:50.824162707 +0000 UTC m=+482.793798737" watchObservedRunningTime="2026-07-07 18:57:50.824448106 +0000 UTC m=+482.794084100" Jul 07 18:57:50.841937 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:50.841873 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-8njz4" podStartSLOduration=1.617564835 podStartE2EDuration="5.841854071s" totalImagesPullingTime="4.224289236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:57:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:57:50.840359327 +0000 UTC m=+482.809995371" watchObservedRunningTime="2026-07-07 18:57:50.841854071 +0000 UTC m=+482.811490087" Jul 07 18:57:50.867481 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:50.867448 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-8njz4"] Jul 07 18:57:52.816702 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:52.816658 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-8njz4" podUID="e1dd952b-fb36-4374-afc5-1dfae52bc60d" containerName="authorino" containerID="cri-o://246818cf3e851d93f8a3687c47c83f8713f036cf93c989de1037db0f501f09ee" gracePeriod=30 Jul 07 18:57:53.064337 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:53.064314 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8njz4" Jul 07 18:57:53.184162 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:53.184071 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tklrl\" (UniqueName: \"kubernetes.io/projected/e1dd952b-fb36-4374-afc5-1dfae52bc60d-kube-api-access-tklrl\") pod \"e1dd952b-fb36-4374-afc5-1dfae52bc60d\" (UID: \"e1dd952b-fb36-4374-afc5-1dfae52bc60d\") " Jul 07 18:57:53.186102 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:53.186070 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1dd952b-fb36-4374-afc5-1dfae52bc60d-kube-api-access-tklrl" (OuterVolumeSpecName: "kube-api-access-tklrl") pod "e1dd952b-fb36-4374-afc5-1dfae52bc60d" (UID: "e1dd952b-fb36-4374-afc5-1dfae52bc60d"). InnerVolumeSpecName "kube-api-access-tklrl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:57:53.284913 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:53.284885 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tklrl\" (UniqueName: \"kubernetes.io/projected/e1dd952b-fb36-4374-afc5-1dfae52bc60d-kube-api-access-tklrl\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:57:53.821840 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:53.821803 2564 generic.go:356] "Generic (PLEG): container finished" podID="e1dd952b-fb36-4374-afc5-1dfae52bc60d" containerID="246818cf3e851d93f8a3687c47c83f8713f036cf93c989de1037db0f501f09ee" exitCode=0 Jul 07 18:57:53.822296 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:53.821851 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-8njz4" Jul 07 18:57:53.822296 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:53.821888 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8njz4" event={"ID":"e1dd952b-fb36-4374-afc5-1dfae52bc60d","Type":"ContainerDied","Data":"246818cf3e851d93f8a3687c47c83f8713f036cf93c989de1037db0f501f09ee"} Jul 07 18:57:53.822296 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:53.821929 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-8njz4" event={"ID":"e1dd952b-fb36-4374-afc5-1dfae52bc60d","Type":"ContainerDied","Data":"6676adb9db0f6b8af79b7d80de79203bada32b016acbb225d7233a4ef6cab901"} Jul 07 18:57:53.822296 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:53.821950 2564 scope.go:117] "RemoveContainer" containerID="246818cf3e851d93f8a3687c47c83f8713f036cf93c989de1037db0f501f09ee" Jul 07 18:57:53.830499 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:53.830482 2564 scope.go:117] "RemoveContainer" containerID="246818cf3e851d93f8a3687c47c83f8713f036cf93c989de1037db0f501f09ee" Jul 07 18:57:53.830734 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:57:53.830718 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"246818cf3e851d93f8a3687c47c83f8713f036cf93c989de1037db0f501f09ee\": container with ID starting with 246818cf3e851d93f8a3687c47c83f8713f036cf93c989de1037db0f501f09ee not found: ID does not exist" containerID="246818cf3e851d93f8a3687c47c83f8713f036cf93c989de1037db0f501f09ee" Jul 07 18:57:53.830776 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:53.830741 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"246818cf3e851d93f8a3687c47c83f8713f036cf93c989de1037db0f501f09ee"} err="failed to get container status \"246818cf3e851d93f8a3687c47c83f8713f036cf93c989de1037db0f501f09ee\": rpc error: code = NotFound desc = could not find container \"246818cf3e851d93f8a3687c47c83f8713f036cf93c989de1037db0f501f09ee\": container with ID starting with 246818cf3e851d93f8a3687c47c83f8713f036cf93c989de1037db0f501f09ee not found: ID does not exist" Jul 07 18:57:53.848533 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:53.848509 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-8njz4"] Jul 07 18:57:53.853295 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:53.853276 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-8njz4"] Jul 07 18:57:54.572934 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:54.572901 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1dd952b-fb36-4374-afc5-1dfae52bc60d" path="/var/lib/kubelet/pods/e1dd952b-fb36-4374-afc5-1dfae52bc60d/volumes" Jul 07 18:57:59.798143 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:59.798113 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s8qcl"] Jul 07 18:57:59.798586 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:59.798301 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-s8qcl" podUID="2a7a9358-2829-403a-b7af-10ffb34e7ea5" containerName="limitador" containerID="cri-o://8a9e667e3776190407e08bd35793c643c6737a12a5ce576adffdccebb62d5df3" gracePeriod=30 Jul 07 18:57:59.799005 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:57:59.798987 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-s8qcl" Jul 07 18:58:00.739556 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:00.739534 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-s8qcl" Jul 07 18:58:00.845962 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:00.845936 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2a7a9358-2829-403a-b7af-10ffb34e7ea5-config-file\") pod \"2a7a9358-2829-403a-b7af-10ffb34e7ea5\" (UID: \"2a7a9358-2829-403a-b7af-10ffb34e7ea5\") " Jul 07 18:58:00.846312 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:00.845999 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6vkz\" (UniqueName: \"kubernetes.io/projected/2a7a9358-2829-403a-b7af-10ffb34e7ea5-kube-api-access-c6vkz\") pod \"2a7a9358-2829-403a-b7af-10ffb34e7ea5\" (UID: \"2a7a9358-2829-403a-b7af-10ffb34e7ea5\") " Jul 07 18:58:00.846312 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:00.846285 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a7a9358-2829-403a-b7af-10ffb34e7ea5-config-file" (OuterVolumeSpecName: "config-file") pod "2a7a9358-2829-403a-b7af-10ffb34e7ea5" (UID: "2a7a9358-2829-403a-b7af-10ffb34e7ea5"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 18:58:00.847924 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:00.847897 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a7a9358-2829-403a-b7af-10ffb34e7ea5-kube-api-access-c6vkz" (OuterVolumeSpecName: "kube-api-access-c6vkz") pod "2a7a9358-2829-403a-b7af-10ffb34e7ea5" (UID: "2a7a9358-2829-403a-b7af-10ffb34e7ea5"). InnerVolumeSpecName "kube-api-access-c6vkz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:58:00.855022 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:00.854998 2564 generic.go:356] "Generic (PLEG): container finished" podID="2a7a9358-2829-403a-b7af-10ffb34e7ea5" containerID="8a9e667e3776190407e08bd35793c643c6737a12a5ce576adffdccebb62d5df3" exitCode=0 Jul 07 18:58:00.855105 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:00.855035 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-s8qcl" Jul 07 18:58:00.855105 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:00.855074 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-s8qcl" event={"ID":"2a7a9358-2829-403a-b7af-10ffb34e7ea5","Type":"ContainerDied","Data":"8a9e667e3776190407e08bd35793c643c6737a12a5ce576adffdccebb62d5df3"} Jul 07 18:58:00.855186 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:00.855128 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-s8qcl" event={"ID":"2a7a9358-2829-403a-b7af-10ffb34e7ea5","Type":"ContainerDied","Data":"fba884674c646e223cdb3c4850ccb3aa40ddee48bfc4b308123aae650e95381e"} Jul 07 18:58:00.855186 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:00.855150 2564 scope.go:117] "RemoveContainer" containerID="8a9e667e3776190407e08bd35793c643c6737a12a5ce576adffdccebb62d5df3" Jul 07 18:58:00.863290 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:00.863271 2564 scope.go:117] "RemoveContainer" containerID="8a9e667e3776190407e08bd35793c643c6737a12a5ce576adffdccebb62d5df3" Jul 07 18:58:00.863533 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:58:00.863514 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a9e667e3776190407e08bd35793c643c6737a12a5ce576adffdccebb62d5df3\": container with ID starting with 8a9e667e3776190407e08bd35793c643c6737a12a5ce576adffdccebb62d5df3 not found: ID does not exist" containerID="8a9e667e3776190407e08bd35793c643c6737a12a5ce576adffdccebb62d5df3" Jul 07 18:58:00.863595 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:00.863541 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a9e667e3776190407e08bd35793c643c6737a12a5ce576adffdccebb62d5df3"} err="failed to get container status \"8a9e667e3776190407e08bd35793c643c6737a12a5ce576adffdccebb62d5df3\": rpc error: code = NotFound desc = could not find container \"8a9e667e3776190407e08bd35793c643c6737a12a5ce576adffdccebb62d5df3\": container with ID starting with 8a9e667e3776190407e08bd35793c643c6737a12a5ce576adffdccebb62d5df3 not found: ID does not exist" Jul 07 18:58:00.876491 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:00.876467 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s8qcl"] Jul 07 18:58:00.889019 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:00.888999 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-s8qcl"] Jul 07 18:58:00.947314 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:00.947295 2564 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2a7a9358-2829-403a-b7af-10ffb34e7ea5-config-file\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:58:00.947314 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:00.947314 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6vkz\" (UniqueName: \"kubernetes.io/projected/2a7a9358-2829-403a-b7af-10ffb34e7ea5-kube-api-access-c6vkz\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:58:02.443159 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:02.443127 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-n4dv8"] Jul 07 18:58:02.443527 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:02.443485 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1dd952b-fb36-4374-afc5-1dfae52bc60d" containerName="authorino" Jul 07 18:58:02.443527 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:02.443497 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1dd952b-fb36-4374-afc5-1dfae52bc60d" containerName="authorino" Jul 07 18:58:02.443527 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:02.443519 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a7a9358-2829-403a-b7af-10ffb34e7ea5" containerName="limitador" Jul 07 18:58:02.443527 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:02.443527 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a7a9358-2829-403a-b7af-10ffb34e7ea5" containerName="limitador" Jul 07 18:58:02.443657 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:02.443581 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1dd952b-fb36-4374-afc5-1dfae52bc60d" containerName="authorino" Jul 07 18:58:02.443657 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:02.443591 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a7a9358-2829-403a-b7af-10ffb34e7ea5" containerName="limitador" Jul 07 18:58:02.447773 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:02.447754 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-n4dv8" Jul 07 18:58:02.449721 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:02.449697 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 07 18:58:02.449840 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:02.449746 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-fglp2\"" Jul 07 18:58:02.454847 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:02.454767 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-n4dv8"] Jul 07 18:58:02.559117 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:02.559092 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w75v\" (UniqueName: \"kubernetes.io/projected/4946fcd5-d821-4a02-bf11-23b7658109f0-kube-api-access-6w75v\") pod \"postgres-69fb4594cf-n4dv8\" (UID: \"4946fcd5-d821-4a02-bf11-23b7658109f0\") " pod="opendatahub/postgres-69fb4594cf-n4dv8" Jul 07 18:58:02.559241 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:02.559160 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4946fcd5-d821-4a02-bf11-23b7658109f0-data\") pod \"postgres-69fb4594cf-n4dv8\" (UID: \"4946fcd5-d821-4a02-bf11-23b7658109f0\") " pod="opendatahub/postgres-69fb4594cf-n4dv8" Jul 07 18:58:02.572634 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:02.572609 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a7a9358-2829-403a-b7af-10ffb34e7ea5" path="/var/lib/kubelet/pods/2a7a9358-2829-403a-b7af-10ffb34e7ea5/volumes" Jul 07 18:58:02.659885 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:02.659858 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w75v\" (UniqueName: \"kubernetes.io/projected/4946fcd5-d821-4a02-bf11-23b7658109f0-kube-api-access-6w75v\") pod \"postgres-69fb4594cf-n4dv8\" (UID: \"4946fcd5-d821-4a02-bf11-23b7658109f0\") " pod="opendatahub/postgres-69fb4594cf-n4dv8" Jul 07 18:58:02.659997 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:02.659947 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4946fcd5-d821-4a02-bf11-23b7658109f0-data\") pod \"postgres-69fb4594cf-n4dv8\" (UID: \"4946fcd5-d821-4a02-bf11-23b7658109f0\") " pod="opendatahub/postgres-69fb4594cf-n4dv8" Jul 07 18:58:02.660353 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:02.660337 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4946fcd5-d821-4a02-bf11-23b7658109f0-data\") pod \"postgres-69fb4594cf-n4dv8\" (UID: \"4946fcd5-d821-4a02-bf11-23b7658109f0\") " pod="opendatahub/postgres-69fb4594cf-n4dv8" Jul 07 18:58:02.668943 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:02.668923 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w75v\" (UniqueName: \"kubernetes.io/projected/4946fcd5-d821-4a02-bf11-23b7658109f0-kube-api-access-6w75v\") pod \"postgres-69fb4594cf-n4dv8\" (UID: \"4946fcd5-d821-4a02-bf11-23b7658109f0\") " pod="opendatahub/postgres-69fb4594cf-n4dv8" Jul 07 18:58:02.758706 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:02.758655 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-n4dv8" Jul 07 18:58:03.098327 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:03.098253 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-n4dv8"] Jul 07 18:58:03.101429 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:58:03.101402 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4946fcd5_d821_4a02_bf11_23b7658109f0.slice/crio-721abbfe10feec7a7c67f4ccfbbd7fc6d8a8a7e0320807a4d13008391565c34b WatchSource:0}: Error finding container 721abbfe10feec7a7c67f4ccfbbd7fc6d8a8a7e0320807a4d13008391565c34b: Status 404 returned error can't find the container with id 721abbfe10feec7a7c67f4ccfbbd7fc6d8a8a7e0320807a4d13008391565c34b Jul 07 18:58:03.875563 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:03.875494 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-n4dv8" event={"ID":"4946fcd5-d821-4a02-bf11-23b7658109f0","Type":"ContainerStarted","Data":"721abbfe10feec7a7c67f4ccfbbd7fc6d8a8a7e0320807a4d13008391565c34b"} Jul 07 18:58:07.897344 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:07.897314 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-n4dv8" event={"ID":"4946fcd5-d821-4a02-bf11-23b7658109f0","Type":"ContainerStarted","Data":"748b8de0adc509104d00af7d7f74e8b275ff476b66d38e1fd0b41f99fb1c1131"} Jul 07 18:58:07.897660 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:07.897489 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-n4dv8" Jul 07 18:58:07.917954 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:07.917910 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-n4dv8" podStartSLOduration=1.201548155 podStartE2EDuration="5.917895538s" totalImagesPullingTime="4.716347383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:58:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:58:07.915574691 +0000 UTC m=+499.885210709" watchObservedRunningTime="2026-07-07 18:58:07.917895538 +0000 UTC m=+499.887531555" Jul 07 18:58:13.929975 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:13.929946 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-n4dv8" Jul 07 18:58:14.476361 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:14.476327 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jt57t"] Jul 07 18:58:14.479794 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:14.479777 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jt57t" Jul 07 18:58:14.488492 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:14.488468 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jt57t"] Jul 07 18:58:14.559931 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:14.559903 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf976\" (UniqueName: \"kubernetes.io/projected/d738f03b-fb3b-47ca-90f4-b1e258b66b44-kube-api-access-tf976\") pod \"authorino-5d4b67c446-jt57t\" (UID: \"d738f03b-fb3b-47ca-90f4-b1e258b66b44\") " pod="kuadrant-system/authorino-5d4b67c446-jt57t" Jul 07 18:58:14.661225 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:14.661176 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf976\" (UniqueName: \"kubernetes.io/projected/d738f03b-fb3b-47ca-90f4-b1e258b66b44-kube-api-access-tf976\") pod \"authorino-5d4b67c446-jt57t\" (UID: \"d738f03b-fb3b-47ca-90f4-b1e258b66b44\") " pod="kuadrant-system/authorino-5d4b67c446-jt57t" Jul 07 18:58:14.670672 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:14.670645 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf976\" (UniqueName: \"kubernetes.io/projected/d738f03b-fb3b-47ca-90f4-b1e258b66b44-kube-api-access-tf976\") pod \"authorino-5d4b67c446-jt57t\" (UID: \"d738f03b-fb3b-47ca-90f4-b1e258b66b44\") " pod="kuadrant-system/authorino-5d4b67c446-jt57t" Jul 07 18:58:14.722851 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:14.722826 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jt57t"] Jul 07 18:58:14.723039 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:14.723028 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jt57t" Jul 07 18:58:14.754392 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:14.754365 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d78dc74b8-72rdq"] Jul 07 18:58:14.758584 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:14.758565 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d78dc74b8-72rdq" Jul 07 18:58:14.765440 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:14.765416 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d78dc74b8-72rdq"] Jul 07 18:58:14.860640 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:14.860615 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jt57t"] Jul 07 18:58:14.862168 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:14.862146 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4rmm\" (UniqueName: \"kubernetes.io/projected/f60e9438-1c45-4862-8b6e-615b728a1bc4-kube-api-access-v4rmm\") pod \"authorino-5d78dc74b8-72rdq\" (UID: \"f60e9438-1c45-4862-8b6e-615b728a1bc4\") " pod="kuadrant-system/authorino-5d78dc74b8-72rdq" Jul 07 18:58:14.862992 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:58:14.862968 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd738f03b_fb3b_47ca_90f4_b1e258b66b44.slice/crio-880a1c6a4aa86753e98df8254c4f034bec6576c1fb0313b99c4d9ed7b4e4534a WatchSource:0}: Error finding container 880a1c6a4aa86753e98df8254c4f034bec6576c1fb0313b99c4d9ed7b4e4534a: Status 404 returned error can't find the container with id 880a1c6a4aa86753e98df8254c4f034bec6576c1fb0313b99c4d9ed7b4e4534a Jul 07 18:58:14.928653 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:14.928625 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jt57t" event={"ID":"d738f03b-fb3b-47ca-90f4-b1e258b66b44","Type":"ContainerStarted","Data":"880a1c6a4aa86753e98df8254c4f034bec6576c1fb0313b99c4d9ed7b4e4534a"} Jul 07 18:58:14.962901 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:14.962877 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4rmm\" (UniqueName: \"kubernetes.io/projected/f60e9438-1c45-4862-8b6e-615b728a1bc4-kube-api-access-v4rmm\") pod \"authorino-5d78dc74b8-72rdq\" (UID: \"f60e9438-1c45-4862-8b6e-615b728a1bc4\") " pod="kuadrant-system/authorino-5d78dc74b8-72rdq" Jul 07 18:58:14.974349 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:14.974330 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4rmm\" (UniqueName: \"kubernetes.io/projected/f60e9438-1c45-4862-8b6e-615b728a1bc4-kube-api-access-v4rmm\") pod \"authorino-5d78dc74b8-72rdq\" (UID: \"f60e9438-1c45-4862-8b6e-615b728a1bc4\") " pod="kuadrant-system/authorino-5d78dc74b8-72rdq" Jul 07 18:58:15.048260 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:15.048160 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d78dc74b8-72rdq"] Jul 07 18:58:15.048481 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:15.048464 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d78dc74b8-72rdq" Jul 07 18:58:15.179726 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:15.179701 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d78dc74b8-72rdq"] Jul 07 18:58:15.181673 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:58:15.181652 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf60e9438_1c45_4862_8b6e_615b728a1bc4.slice/crio-b456cffcf286fd79f145c037afd433b8129156b842a591cd72503855a8228d94 WatchSource:0}: Error finding container b456cffcf286fd79f145c037afd433b8129156b842a591cd72503855a8228d94: Status 404 returned error can't find the container with id b456cffcf286fd79f145c037afd433b8129156b842a591cd72503855a8228d94 Jul 07 18:58:15.934383 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:15.934291 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d78dc74b8-72rdq" event={"ID":"f60e9438-1c45-4862-8b6e-615b728a1bc4","Type":"ContainerStarted","Data":"465ee68edb525324c5a24263dfb5702397d4248544d36f4200f53ea7abd68f14"} Jul 07 18:58:15.934383 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:15.934331 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d78dc74b8-72rdq" event={"ID":"f60e9438-1c45-4862-8b6e-615b728a1bc4","Type":"ContainerStarted","Data":"b456cffcf286fd79f145c037afd433b8129156b842a591cd72503855a8228d94"} Jul 07 18:58:15.934383 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:15.934374 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d78dc74b8-72rdq" podUID="f60e9438-1c45-4862-8b6e-615b728a1bc4" containerName="authorino" containerID="cri-o://465ee68edb525324c5a24263dfb5702397d4248544d36f4200f53ea7abd68f14" gracePeriod=30 Jul 07 18:58:15.935858 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:15.935834 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jt57t" event={"ID":"d738f03b-fb3b-47ca-90f4-b1e258b66b44","Type":"ContainerStarted","Data":"1f27aa8d0e9f166711574990932a5db0be243e9a9b2f61239bab1694d2e198a4"} Jul 07 18:58:15.935989 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:15.935895 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-jt57t" podUID="d738f03b-fb3b-47ca-90f4-b1e258b66b44" containerName="authorino" containerID="cri-o://1f27aa8d0e9f166711574990932a5db0be243e9a9b2f61239bab1694d2e198a4" gracePeriod=30 Jul 07 18:58:15.954177 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:15.954136 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d78dc74b8-72rdq" podStartSLOduration=1.566188352 podStartE2EDuration="1.954123664s" totalImagesPullingTime="387.935312ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:58:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:58:15.952883307 +0000 UTC m=+507.922519326" watchObservedRunningTime="2026-07-07 18:58:15.954123664 +0000 UTC m=+507.923759681" Jul 07 18:58:15.972476 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:15.972438 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-jt57t" podStartSLOduration=1.654690575 podStartE2EDuration="1.972426338s" totalImagesPullingTime="317.735763ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:58:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:58:15.971357593 +0000 UTC m=+507.940993629" watchObservedRunningTime="2026-07-07 18:58:15.972426338 +0000 UTC m=+507.942062355" Jul 07 18:58:16.212736 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.212713 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jt57t" Jul 07 18:58:16.215005 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.214987 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d78dc74b8-72rdq" Jul 07 18:58:16.273748 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.273720 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tf976\" (UniqueName: \"kubernetes.io/projected/d738f03b-fb3b-47ca-90f4-b1e258b66b44-kube-api-access-tf976\") pod \"d738f03b-fb3b-47ca-90f4-b1e258b66b44\" (UID: \"d738f03b-fb3b-47ca-90f4-b1e258b66b44\") " Jul 07 18:58:16.273748 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.273750 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4rmm\" (UniqueName: \"kubernetes.io/projected/f60e9438-1c45-4862-8b6e-615b728a1bc4-kube-api-access-v4rmm\") pod \"f60e9438-1c45-4862-8b6e-615b728a1bc4\" (UID: \"f60e9438-1c45-4862-8b6e-615b728a1bc4\") " Jul 07 18:58:16.275805 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.275780 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f60e9438-1c45-4862-8b6e-615b728a1bc4-kube-api-access-v4rmm" (OuterVolumeSpecName: "kube-api-access-v4rmm") pod "f60e9438-1c45-4862-8b6e-615b728a1bc4" (UID: "f60e9438-1c45-4862-8b6e-615b728a1bc4"). InnerVolumeSpecName "kube-api-access-v4rmm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:58:16.275904 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.275813 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d738f03b-fb3b-47ca-90f4-b1e258b66b44-kube-api-access-tf976" (OuterVolumeSpecName: "kube-api-access-tf976") pod "d738f03b-fb3b-47ca-90f4-b1e258b66b44" (UID: "d738f03b-fb3b-47ca-90f4-b1e258b66b44"). InnerVolumeSpecName "kube-api-access-tf976". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:58:16.375078 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.375040 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tf976\" (UniqueName: \"kubernetes.io/projected/d738f03b-fb3b-47ca-90f4-b1e258b66b44-kube-api-access-tf976\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:58:16.375078 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.375074 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4rmm\" (UniqueName: \"kubernetes.io/projected/f60e9438-1c45-4862-8b6e-615b728a1bc4-kube-api-access-v4rmm\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:58:16.941177 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.941140 2564 generic.go:356] "Generic (PLEG): container finished" podID="d738f03b-fb3b-47ca-90f4-b1e258b66b44" containerID="1f27aa8d0e9f166711574990932a5db0be243e9a9b2f61239bab1694d2e198a4" exitCode=0 Jul 07 18:58:16.941340 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.941214 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-jt57t" Jul 07 18:58:16.941340 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.941227 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jt57t" event={"ID":"d738f03b-fb3b-47ca-90f4-b1e258b66b44","Type":"ContainerDied","Data":"1f27aa8d0e9f166711574990932a5db0be243e9a9b2f61239bab1694d2e198a4"} Jul 07 18:58:16.941340 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.941273 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-jt57t" event={"ID":"d738f03b-fb3b-47ca-90f4-b1e258b66b44","Type":"ContainerDied","Data":"880a1c6a4aa86753e98df8254c4f034bec6576c1fb0313b99c4d9ed7b4e4534a"} Jul 07 18:58:16.941340 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.941291 2564 scope.go:117] "RemoveContainer" containerID="1f27aa8d0e9f166711574990932a5db0be243e9a9b2f61239bab1694d2e198a4" Jul 07 18:58:16.942831 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.942806 2564 generic.go:356] "Generic (PLEG): container finished" podID="f60e9438-1c45-4862-8b6e-615b728a1bc4" containerID="465ee68edb525324c5a24263dfb5702397d4248544d36f4200f53ea7abd68f14" exitCode=0 Jul 07 18:58:16.942949 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.942853 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d78dc74b8-72rdq" Jul 07 18:58:16.942949 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.942858 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d78dc74b8-72rdq" event={"ID":"f60e9438-1c45-4862-8b6e-615b728a1bc4","Type":"ContainerDied","Data":"465ee68edb525324c5a24263dfb5702397d4248544d36f4200f53ea7abd68f14"} Jul 07 18:58:16.942949 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.942888 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d78dc74b8-72rdq" event={"ID":"f60e9438-1c45-4862-8b6e-615b728a1bc4","Type":"ContainerDied","Data":"b456cffcf286fd79f145c037afd433b8129156b842a591cd72503855a8228d94"} Jul 07 18:58:16.949572 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.949555 2564 scope.go:117] "RemoveContainer" containerID="1f27aa8d0e9f166711574990932a5db0be243e9a9b2f61239bab1694d2e198a4" Jul 07 18:58:16.949797 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:58:16.949780 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f27aa8d0e9f166711574990932a5db0be243e9a9b2f61239bab1694d2e198a4\": container with ID starting with 1f27aa8d0e9f166711574990932a5db0be243e9a9b2f61239bab1694d2e198a4 not found: ID does not exist" containerID="1f27aa8d0e9f166711574990932a5db0be243e9a9b2f61239bab1694d2e198a4" Jul 07 18:58:16.949842 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.949805 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f27aa8d0e9f166711574990932a5db0be243e9a9b2f61239bab1694d2e198a4"} err="failed to get container status \"1f27aa8d0e9f166711574990932a5db0be243e9a9b2f61239bab1694d2e198a4\": rpc error: code = NotFound desc = could not find container \"1f27aa8d0e9f166711574990932a5db0be243e9a9b2f61239bab1694d2e198a4\": container with ID starting with 1f27aa8d0e9f166711574990932a5db0be243e9a9b2f61239bab1694d2e198a4 not found: ID does not exist" Jul 07 18:58:16.949842 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.949822 2564 scope.go:117] "RemoveContainer" containerID="465ee68edb525324c5a24263dfb5702397d4248544d36f4200f53ea7abd68f14" Jul 07 18:58:16.958116 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.957898 2564 scope.go:117] "RemoveContainer" containerID="465ee68edb525324c5a24263dfb5702397d4248544d36f4200f53ea7abd68f14" Jul 07 18:58:16.958282 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:58:16.958258 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"465ee68edb525324c5a24263dfb5702397d4248544d36f4200f53ea7abd68f14\": container with ID starting with 465ee68edb525324c5a24263dfb5702397d4248544d36f4200f53ea7abd68f14 not found: ID does not exist" containerID="465ee68edb525324c5a24263dfb5702397d4248544d36f4200f53ea7abd68f14" Jul 07 18:58:16.958361 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.958292 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"465ee68edb525324c5a24263dfb5702397d4248544d36f4200f53ea7abd68f14"} err="failed to get container status \"465ee68edb525324c5a24263dfb5702397d4248544d36f4200f53ea7abd68f14\": rpc error: code = NotFound desc = could not find container \"465ee68edb525324c5a24263dfb5702397d4248544d36f4200f53ea7abd68f14\": container with ID starting with 465ee68edb525324c5a24263dfb5702397d4248544d36f4200f53ea7abd68f14 not found: ID does not exist" Jul 07 18:58:16.958361 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.958320 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jt57t"] Jul 07 18:58:16.960084 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.960066 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-jt57t"] Jul 07 18:58:16.972974 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.972951 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d78dc74b8-72rdq"] Jul 07 18:58:16.977919 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:16.977901 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d78dc74b8-72rdq"] Jul 07 18:58:18.572801 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:18.572765 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d738f03b-fb3b-47ca-90f4-b1e258b66b44" path="/var/lib/kubelet/pods/d738f03b-fb3b-47ca-90f4-b1e258b66b44/volumes" Jul 07 18:58:18.573311 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:18.573083 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f60e9438-1c45-4862-8b6e-615b728a1bc4" path="/var/lib/kubelet/pods/f60e9438-1c45-4862-8b6e-615b728a1bc4/volumes" Jul 07 18:58:18.941549 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:18.941520 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m2wd4"] Jul 07 18:58:18.941730 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:18.941708 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-m2wd4" podUID="281dcb4a-48c1-4720-9059-4d03d7e4eea1" containerName="authorino" containerID="cri-o://8484b06c164f58b4c60dcbeff681e9b8743a341f982c5fa48c14739612df23e4" gracePeriod=30 Jul 07 18:58:19.227963 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:19.227935 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-m2wd4" Jul 07 18:58:19.298662 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:19.298635 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nq94r\" (UniqueName: \"kubernetes.io/projected/281dcb4a-48c1-4720-9059-4d03d7e4eea1-kube-api-access-nq94r\") pod \"281dcb4a-48c1-4720-9059-4d03d7e4eea1\" (UID: \"281dcb4a-48c1-4720-9059-4d03d7e4eea1\") " Jul 07 18:58:19.300579 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:19.300543 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/281dcb4a-48c1-4720-9059-4d03d7e4eea1-kube-api-access-nq94r" (OuterVolumeSpecName: "kube-api-access-nq94r") pod "281dcb4a-48c1-4720-9059-4d03d7e4eea1" (UID: "281dcb4a-48c1-4720-9059-4d03d7e4eea1"). InnerVolumeSpecName "kube-api-access-nq94r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:58:19.399753 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:19.399726 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nq94r\" (UniqueName: \"kubernetes.io/projected/281dcb4a-48c1-4720-9059-4d03d7e4eea1-kube-api-access-nq94r\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:58:19.960016 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:19.959982 2564 generic.go:356] "Generic (PLEG): container finished" podID="281dcb4a-48c1-4720-9059-4d03d7e4eea1" containerID="8484b06c164f58b4c60dcbeff681e9b8743a341f982c5fa48c14739612df23e4" exitCode=0 Jul 07 18:58:19.960443 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:19.960033 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-m2wd4" Jul 07 18:58:19.960443 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:19.960039 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-m2wd4" event={"ID":"281dcb4a-48c1-4720-9059-4d03d7e4eea1","Type":"ContainerDied","Data":"8484b06c164f58b4c60dcbeff681e9b8743a341f982c5fa48c14739612df23e4"} Jul 07 18:58:19.960443 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:19.960080 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-m2wd4" event={"ID":"281dcb4a-48c1-4720-9059-4d03d7e4eea1","Type":"ContainerDied","Data":"33cc5a6ff1cfc5213fce58bdae30edc449d44aa40644b7393cdc2d2f6c1cd416"} Jul 07 18:58:19.960443 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:19.960097 2564 scope.go:117] "RemoveContainer" containerID="8484b06c164f58b4c60dcbeff681e9b8743a341f982c5fa48c14739612df23e4" Jul 07 18:58:19.967924 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:19.967827 2564 scope.go:117] "RemoveContainer" containerID="8484b06c164f58b4c60dcbeff681e9b8743a341f982c5fa48c14739612df23e4" Jul 07 18:58:19.968109 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:58:19.968092 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8484b06c164f58b4c60dcbeff681e9b8743a341f982c5fa48c14739612df23e4\": container with ID starting with 8484b06c164f58b4c60dcbeff681e9b8743a341f982c5fa48c14739612df23e4 not found: ID does not exist" containerID="8484b06c164f58b4c60dcbeff681e9b8743a341f982c5fa48c14739612df23e4" Jul 07 18:58:19.968178 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:19.968117 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8484b06c164f58b4c60dcbeff681e9b8743a341f982c5fa48c14739612df23e4"} err="failed to get container status \"8484b06c164f58b4c60dcbeff681e9b8743a341f982c5fa48c14739612df23e4\": rpc error: code = NotFound desc = could not find container \"8484b06c164f58b4c60dcbeff681e9b8743a341f982c5fa48c14739612df23e4\": container with ID starting with 8484b06c164f58b4c60dcbeff681e9b8743a341f982c5fa48c14739612df23e4 not found: ID does not exist" Jul 07 18:58:19.987528 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:19.987508 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m2wd4"] Jul 07 18:58:19.989930 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:19.989909 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-m2wd4"] Jul 07 18:58:20.572744 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:20.572712 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="281dcb4a-48c1-4720-9059-4d03d7e4eea1" path="/var/lib/kubelet/pods/281dcb4a-48c1-4720-9059-4d03d7e4eea1/volumes" Jul 07 18:58:23.397071 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.397038 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d9575f9c8-4b89b"] Jul 07 18:58:23.397463 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.397404 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="281dcb4a-48c1-4720-9059-4d03d7e4eea1" containerName="authorino" Jul 07 18:58:23.397463 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.397416 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="281dcb4a-48c1-4720-9059-4d03d7e4eea1" containerName="authorino" Jul 07 18:58:23.397463 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.397438 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d738f03b-fb3b-47ca-90f4-b1e258b66b44" containerName="authorino" Jul 07 18:58:23.397463 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.397444 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d738f03b-fb3b-47ca-90f4-b1e258b66b44" containerName="authorino" Jul 07 18:58:23.397463 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.397451 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f60e9438-1c45-4862-8b6e-615b728a1bc4" containerName="authorino" Jul 07 18:58:23.397463 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.397456 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f60e9438-1c45-4862-8b6e-615b728a1bc4" containerName="authorino" Jul 07 18:58:23.397664 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.397515 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d738f03b-fb3b-47ca-90f4-b1e258b66b44" containerName="authorino" Jul 07 18:58:23.397664 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.397523 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f60e9438-1c45-4862-8b6e-615b728a1bc4" containerName="authorino" Jul 07 18:58:23.397664 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.397531 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="281dcb4a-48c1-4720-9059-4d03d7e4eea1" containerName="authorino" Jul 07 18:58:23.404734 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.404711 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d9575f9c8-4b89b" Jul 07 18:58:23.407368 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.407348 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 07 18:58:23.407368 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.407360 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kwg5x\"" Jul 07 18:58:23.407550 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.407383 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 07 18:58:23.414340 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.414317 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d9575f9c8-4b89b"] Jul 07 18:58:23.434128 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.434107 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f16970a3-1e69-41f0-9815-1b4af45010bd-webhook-cert\") pod \"maas-controller-d9575f9c8-4b89b\" (UID: \"f16970a3-1e69-41f0-9815-1b4af45010bd\") " pod="opendatahub/maas-controller-d9575f9c8-4b89b" Jul 07 18:58:23.434241 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.434141 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfffx\" (UniqueName: \"kubernetes.io/projected/f16970a3-1e69-41f0-9815-1b4af45010bd-kube-api-access-pfffx\") pod \"maas-controller-d9575f9c8-4b89b\" (UID: \"f16970a3-1e69-41f0-9815-1b4af45010bd\") " pod="opendatahub/maas-controller-d9575f9c8-4b89b" Jul 07 18:58:23.534950 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.534924 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f16970a3-1e69-41f0-9815-1b4af45010bd-webhook-cert\") pod \"maas-controller-d9575f9c8-4b89b\" (UID: \"f16970a3-1e69-41f0-9815-1b4af45010bd\") " pod="opendatahub/maas-controller-d9575f9c8-4b89b" Jul 07 18:58:23.535068 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.534959 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfffx\" (UniqueName: \"kubernetes.io/projected/f16970a3-1e69-41f0-9815-1b4af45010bd-kube-api-access-pfffx\") pod \"maas-controller-d9575f9c8-4b89b\" (UID: \"f16970a3-1e69-41f0-9815-1b4af45010bd\") " pod="opendatahub/maas-controller-d9575f9c8-4b89b" Jul 07 18:58:23.535136 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:58:23.535083 2564 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 07 18:58:23.535191 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:58:23.535160 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f16970a3-1e69-41f0-9815-1b4af45010bd-webhook-cert podName:f16970a3-1e69-41f0-9815-1b4af45010bd nodeName:}" failed. No retries permitted until 2026-07-07 18:58:24.035137949 +0000 UTC m=+516.004773956 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/f16970a3-1e69-41f0-9815-1b4af45010bd-webhook-cert") pod "maas-controller-d9575f9c8-4b89b" (UID: "f16970a3-1e69-41f0-9815-1b4af45010bd") : secret "maas-controller-webhook-cert" not found Jul 07 18:58:23.547721 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.547697 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfffx\" (UniqueName: \"kubernetes.io/projected/f16970a3-1e69-41f0-9815-1b4af45010bd-kube-api-access-pfffx\") pod \"maas-controller-d9575f9c8-4b89b\" (UID: \"f16970a3-1e69-41f0-9815-1b4af45010bd\") " pod="opendatahub/maas-controller-d9575f9c8-4b89b" Jul 07 18:58:23.595247 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.595221 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-645cb998cd-klzwt"] Jul 07 18:58:23.598399 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.598385 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-645cb998cd-klzwt" Jul 07 18:58:23.611567 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.611547 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-645cb998cd-klzwt"] Jul 07 18:58:23.635991 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.635972 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4e026420-a6eb-475e-8f43-4fd02c884819-webhook-cert\") pod \"maas-controller-645cb998cd-klzwt\" (UID: \"4e026420-a6eb-475e-8f43-4fd02c884819\") " pod="opendatahub/maas-controller-645cb998cd-klzwt" Jul 07 18:58:23.636087 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.635998 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj5rx\" (UniqueName: \"kubernetes.io/projected/4e026420-a6eb-475e-8f43-4fd02c884819-kube-api-access-vj5rx\") pod \"maas-controller-645cb998cd-klzwt\" (UID: \"4e026420-a6eb-475e-8f43-4fd02c884819\") " pod="opendatahub/maas-controller-645cb998cd-klzwt" Jul 07 18:58:23.737187 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.737166 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4e026420-a6eb-475e-8f43-4fd02c884819-webhook-cert\") pod \"maas-controller-645cb998cd-klzwt\" (UID: \"4e026420-a6eb-475e-8f43-4fd02c884819\") " pod="opendatahub/maas-controller-645cb998cd-klzwt" Jul 07 18:58:23.737311 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.737209 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj5rx\" (UniqueName: \"kubernetes.io/projected/4e026420-a6eb-475e-8f43-4fd02c884819-kube-api-access-vj5rx\") pod \"maas-controller-645cb998cd-klzwt\" (UID: \"4e026420-a6eb-475e-8f43-4fd02c884819\") " pod="opendatahub/maas-controller-645cb998cd-klzwt" Jul 07 18:58:23.737311 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:58:23.737298 2564 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 07 18:58:23.737390 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:58:23.737381 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e026420-a6eb-475e-8f43-4fd02c884819-webhook-cert podName:4e026420-a6eb-475e-8f43-4fd02c884819 nodeName:}" failed. No retries permitted until 2026-07-07 18:58:24.237356914 +0000 UTC m=+516.206992920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/4e026420-a6eb-475e-8f43-4fd02c884819-webhook-cert") pod "maas-controller-645cb998cd-klzwt" (UID: "4e026420-a6eb-475e-8f43-4fd02c884819") : secret "maas-controller-webhook-cert" not found Jul 07 18:58:23.746713 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:23.746688 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj5rx\" (UniqueName: \"kubernetes.io/projected/4e026420-a6eb-475e-8f43-4fd02c884819-kube-api-access-vj5rx\") pod \"maas-controller-645cb998cd-klzwt\" (UID: \"4e026420-a6eb-475e-8f43-4fd02c884819\") " pod="opendatahub/maas-controller-645cb998cd-klzwt" Jul 07 18:58:24.040512 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:24.040432 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f16970a3-1e69-41f0-9815-1b4af45010bd-webhook-cert\") pod \"maas-controller-d9575f9c8-4b89b\" (UID: \"f16970a3-1e69-41f0-9815-1b4af45010bd\") " pod="opendatahub/maas-controller-d9575f9c8-4b89b" Jul 07 18:58:24.042543 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:24.042523 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f16970a3-1e69-41f0-9815-1b4af45010bd-webhook-cert\") pod \"maas-controller-d9575f9c8-4b89b\" (UID: \"f16970a3-1e69-41f0-9815-1b4af45010bd\") " pod="opendatahub/maas-controller-d9575f9c8-4b89b" Jul 07 18:58:24.241787 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:24.241754 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4e026420-a6eb-475e-8f43-4fd02c884819-webhook-cert\") pod \"maas-controller-645cb998cd-klzwt\" (UID: \"4e026420-a6eb-475e-8f43-4fd02c884819\") " pod="opendatahub/maas-controller-645cb998cd-klzwt" Jul 07 18:58:24.244129 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:24.244100 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4e026420-a6eb-475e-8f43-4fd02c884819-webhook-cert\") pod \"maas-controller-645cb998cd-klzwt\" (UID: \"4e026420-a6eb-475e-8f43-4fd02c884819\") " pod="opendatahub/maas-controller-645cb998cd-klzwt" Jul 07 18:58:24.315416 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:24.315353 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d9575f9c8-4b89b" Jul 07 18:58:24.444962 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:24.444937 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d9575f9c8-4b89b"] Jul 07 18:58:24.446511 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:58:24.446487 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf16970a3_1e69_41f0_9815_1b4af45010bd.slice/crio-b67fd458db0ba44cb77f80de37f8db3b028a4c851ca1ddd94edf460d64650bd3 WatchSource:0}: Error finding container b67fd458db0ba44cb77f80de37f8db3b028a4c851ca1ddd94edf460d64650bd3: Status 404 returned error can't find the container with id b67fd458db0ba44cb77f80de37f8db3b028a4c851ca1ddd94edf460d64650bd3 Jul 07 18:58:24.508653 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:24.508634 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-645cb998cd-klzwt" Jul 07 18:58:24.640555 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:24.640523 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-645cb998cd-klzwt"] Jul 07 18:58:24.642732 ip-10-0-135-81 kubenswrapper[2564]: W0707 18:58:24.642707 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e026420_a6eb_475e_8f43_4fd02c884819.slice/crio-a83121a6d9c0932263617d33d36c572c96958dc0b143ad4025a6ef2e48242402 WatchSource:0}: Error finding container a83121a6d9c0932263617d33d36c572c96958dc0b143ad4025a6ef2e48242402: Status 404 returned error can't find the container with id a83121a6d9c0932263617d33d36c572c96958dc0b143ad4025a6ef2e48242402 Jul 07 18:58:24.985260 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:24.985226 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d9575f9c8-4b89b" event={"ID":"f16970a3-1e69-41f0-9815-1b4af45010bd","Type":"ContainerStarted","Data":"b67fd458db0ba44cb77f80de37f8db3b028a4c851ca1ddd94edf460d64650bd3"} Jul 07 18:58:24.986323 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:24.986297 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-645cb998cd-klzwt" event={"ID":"4e026420-a6eb-475e-8f43-4fd02c884819","Type":"ContainerStarted","Data":"a83121a6d9c0932263617d33d36c572c96958dc0b143ad4025a6ef2e48242402"} Jul 07 18:58:26.997755 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:26.997717 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d9575f9c8-4b89b" event={"ID":"f16970a3-1e69-41f0-9815-1b4af45010bd","Type":"ContainerStarted","Data":"2a1f12c44d1abca83f1c61c172a566d58f0b8fdfb88d050eba83a3be00325f47"} Jul 07 18:58:26.998285 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:26.997830 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d9575f9c8-4b89b" Jul 07 18:58:26.999269 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:26.999249 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-645cb998cd-klzwt" event={"ID":"4e026420-a6eb-475e-8f43-4fd02c884819","Type":"ContainerStarted","Data":"1c1b4d87c0e44c7bb32867f658161f9e99f1d1e68fff7a9b05ff52db3035efe9"} Jul 07 18:58:26.999340 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:26.999294 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-645cb998cd-klzwt" Jul 07 18:58:27.026279 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:27.026233 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d9575f9c8-4b89b" podStartSLOduration=1.842193829 podStartE2EDuration="4.026221267s" totalImagesPullingTime="2.184027438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:58:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:58:27.024469519 +0000 UTC m=+518.994105537" watchObservedRunningTime="2026-07-07 18:58:27.026221267 +0000 UTC m=+518.995857283" Jul 07 18:58:27.057851 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:27.057804 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-645cb998cd-klzwt" podStartSLOduration=2.080869308 podStartE2EDuration="4.057789222s" totalImagesPullingTime="1.976919914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 18:58:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 18:58:27.056682962 +0000 UTC m=+519.026319000" watchObservedRunningTime="2026-07-07 18:58:27.057789222 +0000 UTC m=+519.027425257" Jul 07 18:58:38.010622 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:38.010592 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d9575f9c8-4b89b" Jul 07 18:58:38.011053 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:38.010785 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-645cb998cd-klzwt" Jul 07 18:58:38.085640 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:38.085609 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d9575f9c8-4b89b"] Jul 07 18:58:38.085858 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:38.085834 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-d9575f9c8-4b89b" podUID="f16970a3-1e69-41f0-9815-1b4af45010bd" containerName="manager" containerID="cri-o://2a1f12c44d1abca83f1c61c172a566d58f0b8fdfb88d050eba83a3be00325f47" gracePeriod=30 Jul 07 18:58:38.325364 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:38.325337 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d9575f9c8-4b89b" Jul 07 18:58:38.464771 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:38.464742 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pfffx\" (UniqueName: \"kubernetes.io/projected/f16970a3-1e69-41f0-9815-1b4af45010bd-kube-api-access-pfffx\") pod \"f16970a3-1e69-41f0-9815-1b4af45010bd\" (UID: \"f16970a3-1e69-41f0-9815-1b4af45010bd\") " Jul 07 18:58:38.464943 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:38.464790 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f16970a3-1e69-41f0-9815-1b4af45010bd-webhook-cert\") pod \"f16970a3-1e69-41f0-9815-1b4af45010bd\" (UID: \"f16970a3-1e69-41f0-9815-1b4af45010bd\") " Jul 07 18:58:38.466837 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:38.466807 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f16970a3-1e69-41f0-9815-1b4af45010bd-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f16970a3-1e69-41f0-9815-1b4af45010bd" (UID: "f16970a3-1e69-41f0-9815-1b4af45010bd"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 18:58:38.466945 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:38.466873 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f16970a3-1e69-41f0-9815-1b4af45010bd-kube-api-access-pfffx" (OuterVolumeSpecName: "kube-api-access-pfffx") pod "f16970a3-1e69-41f0-9815-1b4af45010bd" (UID: "f16970a3-1e69-41f0-9815-1b4af45010bd"). InnerVolumeSpecName "kube-api-access-pfffx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 18:58:38.565642 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:38.565564 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfffx\" (UniqueName: \"kubernetes.io/projected/f16970a3-1e69-41f0-9815-1b4af45010bd-kube-api-access-pfffx\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:58:38.565642 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:38.565588 2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f16970a3-1e69-41f0-9815-1b4af45010bd-webhook-cert\") on node \"ip-10-0-135-81.ec2.internal\" DevicePath \"\"" Jul 07 18:58:39.056701 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:39.056668 2564 generic.go:356] "Generic (PLEG): container finished" podID="f16970a3-1e69-41f0-9815-1b4af45010bd" containerID="2a1f12c44d1abca83f1c61c172a566d58f0b8fdfb88d050eba83a3be00325f47" exitCode=0 Jul 07 18:58:39.057107 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:39.056713 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d9575f9c8-4b89b" Jul 07 18:58:39.057107 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:39.056713 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d9575f9c8-4b89b" event={"ID":"f16970a3-1e69-41f0-9815-1b4af45010bd","Type":"ContainerDied","Data":"2a1f12c44d1abca83f1c61c172a566d58f0b8fdfb88d050eba83a3be00325f47"} Jul 07 18:58:39.057107 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:39.056769 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d9575f9c8-4b89b" event={"ID":"f16970a3-1e69-41f0-9815-1b4af45010bd","Type":"ContainerDied","Data":"b67fd458db0ba44cb77f80de37f8db3b028a4c851ca1ddd94edf460d64650bd3"} Jul 07 18:58:39.057107 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:39.056793 2564 scope.go:117] "RemoveContainer" containerID="2a1f12c44d1abca83f1c61c172a566d58f0b8fdfb88d050eba83a3be00325f47" Jul 07 18:58:39.065462 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:39.065445 2564 scope.go:117] "RemoveContainer" containerID="2a1f12c44d1abca83f1c61c172a566d58f0b8fdfb88d050eba83a3be00325f47" Jul 07 18:58:39.065704 ip-10-0-135-81 kubenswrapper[2564]: E0707 18:58:39.065688 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a1f12c44d1abca83f1c61c172a566d58f0b8fdfb88d050eba83a3be00325f47\": container with ID starting with 2a1f12c44d1abca83f1c61c172a566d58f0b8fdfb88d050eba83a3be00325f47 not found: ID does not exist" containerID="2a1f12c44d1abca83f1c61c172a566d58f0b8fdfb88d050eba83a3be00325f47" Jul 07 18:58:39.065752 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:39.065712 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a1f12c44d1abca83f1c61c172a566d58f0b8fdfb88d050eba83a3be00325f47"} err="failed to get container status \"2a1f12c44d1abca83f1c61c172a566d58f0b8fdfb88d050eba83a3be00325f47\": rpc error: code = NotFound desc = could not find container \"2a1f12c44d1abca83f1c61c172a566d58f0b8fdfb88d050eba83a3be00325f47\": container with ID starting with 2a1f12c44d1abca83f1c61c172a566d58f0b8fdfb88d050eba83a3be00325f47 not found: ID does not exist" Jul 07 18:58:39.082885 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:39.082858 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d9575f9c8-4b89b"] Jul 07 18:58:39.087365 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:39.087343 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d9575f9c8-4b89b"] Jul 07 18:58:40.573055 ip-10-0-135-81 kubenswrapper[2564]: I0707 18:58:40.572982 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f16970a3-1e69-41f0-9815-1b4af45010bd" path="/var/lib/kubelet/pods/f16970a3-1e69-41f0-9815-1b4af45010bd/volumes" Jul 07 19:08:49.607453 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:08:49.607421 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-645cb998cd-klzwt_4e026420-a6eb-475e-8f43-4fd02c884819/manager/0.log" Jul 07 19:08:49.980080 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:08:49.980050 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr_5f7c7b22-3588-4137-ba9f-2dd8d8889ae6/copy-manifests/0.log" Jul 07 19:08:49.987723 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:08:49.987692 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr_5f7c7b22-3588-4137-ba9f-2dd8d8889ae6/manager/0.log" Jul 07 19:08:55.408756 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:08:55.408727 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-n4dv8_4946fcd5-d821-4a02-bf11-23b7658109f0/postgres/0.log" Jul 07 19:08:58.137710 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:08:58.137681 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85b6b9f996-bdq8n_c93fe07d-9362-4713-9dee-e9217aa394e3/kube-auth-proxy/0.log" Jul 07 19:09:02.424819 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:02.424783 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zqqk9/must-gather-bzt7q"] Jul 07 19:09:02.425358 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:02.425338 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f16970a3-1e69-41f0-9815-1b4af45010bd" containerName="manager" Jul 07 19:09:02.425438 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:02.425361 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f16970a3-1e69-41f0-9815-1b4af45010bd" containerName="manager" Jul 07 19:09:02.425495 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:02.425485 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f16970a3-1e69-41f0-9815-1b4af45010bd" containerName="manager" Jul 07 19:09:02.428582 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:02.428557 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zqqk9/must-gather-bzt7q" Jul 07 19:09:02.431564 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:02.431545 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zqqk9\"/\"default-dockercfg-jfdgs\"" Jul 07 19:09:02.431672 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:02.431563 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zqqk9\"/\"kube-root-ca.crt\"" Jul 07 19:09:02.432424 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:02.432406 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zqqk9\"/\"openshift-service-ca.crt\"" Jul 07 19:09:02.455797 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:02.455772 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zqqk9/must-gather-bzt7q"] Jul 07 19:09:02.522980 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:02.522933 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/74946706-214e-45e8-8661-4f5e3e1b17f3-must-gather-output\") pod \"must-gather-bzt7q\" (UID: \"74946706-214e-45e8-8661-4f5e3e1b17f3\") " pod="openshift-must-gather-zqqk9/must-gather-bzt7q" Jul 07 19:09:02.523174 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:02.523016 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6zmd\" (UniqueName: \"kubernetes.io/projected/74946706-214e-45e8-8661-4f5e3e1b17f3-kube-api-access-r6zmd\") pod \"must-gather-bzt7q\" (UID: \"74946706-214e-45e8-8661-4f5e3e1b17f3\") " pod="openshift-must-gather-zqqk9/must-gather-bzt7q" Jul 07 19:09:02.624054 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:02.624026 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6zmd\" (UniqueName: \"kubernetes.io/projected/74946706-214e-45e8-8661-4f5e3e1b17f3-kube-api-access-r6zmd\") pod \"must-gather-bzt7q\" (UID: \"74946706-214e-45e8-8661-4f5e3e1b17f3\") " pod="openshift-must-gather-zqqk9/must-gather-bzt7q" Jul 07 19:09:02.624245 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:02.624092 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/74946706-214e-45e8-8661-4f5e3e1b17f3-must-gather-output\") pod \"must-gather-bzt7q\" (UID: \"74946706-214e-45e8-8661-4f5e3e1b17f3\") " pod="openshift-must-gather-zqqk9/must-gather-bzt7q" Jul 07 19:09:02.624420 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:02.624404 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/74946706-214e-45e8-8661-4f5e3e1b17f3-must-gather-output\") pod \"must-gather-bzt7q\" (UID: \"74946706-214e-45e8-8661-4f5e3e1b17f3\") " pod="openshift-must-gather-zqqk9/must-gather-bzt7q" Jul 07 19:09:02.635621 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:02.635597 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6zmd\" (UniqueName: \"kubernetes.io/projected/74946706-214e-45e8-8661-4f5e3e1b17f3-kube-api-access-r6zmd\") pod \"must-gather-bzt7q\" (UID: \"74946706-214e-45e8-8661-4f5e3e1b17f3\") " pod="openshift-must-gather-zqqk9/must-gather-bzt7q" Jul 07 19:09:02.737399 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:02.737358 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zqqk9/must-gather-bzt7q" Jul 07 19:09:02.868668 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:02.868622 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zqqk9/must-gather-bzt7q"] Jul 07 19:09:02.870761 ip-10-0-135-81 kubenswrapper[2564]: W0707 19:09:02.870732 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod74946706_214e_45e8_8661_4f5e3e1b17f3.slice/crio-56fa8b23aa507ac776bad711e76698a0139464fa5a56ba240cb63ba5aa5b9799 WatchSource:0}: Error finding container 56fa8b23aa507ac776bad711e76698a0139464fa5a56ba240cb63ba5aa5b9799: Status 404 returned error can't find the container with id 56fa8b23aa507ac776bad711e76698a0139464fa5a56ba240cb63ba5aa5b9799 Jul 07 19:09:02.872631 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:02.872612 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 19:09:03.821212 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:03.821160 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zqqk9/must-gather-bzt7q" event={"ID":"74946706-214e-45e8-8661-4f5e3e1b17f3","Type":"ContainerStarted","Data":"56fa8b23aa507ac776bad711e76698a0139464fa5a56ba240cb63ba5aa5b9799"} Jul 07 19:09:04.828577 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:04.828538 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zqqk9/must-gather-bzt7q" event={"ID":"74946706-214e-45e8-8661-4f5e3e1b17f3","Type":"ContainerStarted","Data":"bc682e8f362c67ac8c09752c66eed8555e7a037052053084f309366a0fd59895"} Jul 07 19:09:04.829066 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:04.828590 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zqqk9/must-gather-bzt7q" event={"ID":"74946706-214e-45e8-8661-4f5e3e1b17f3","Type":"ContainerStarted","Data":"a9ccc1027d7fc126ee4c100f9c7bf4aa61729ccd668be2474894343534ef271f"} Jul 07 19:09:04.849535 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:04.849483 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zqqk9/must-gather-bzt7q" podStartSLOduration=1.745448135 podStartE2EDuration="2.849468698s" totalImagesPullingTime="1.104020563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 19:09:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:09:04.846647159 +0000 UTC m=+1156.816283201" watchObservedRunningTime="2026-07-07 19:09:04.849468698 +0000 UTC m=+1156.819104715" Jul 07 19:09:05.631023 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:05.630990 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ffm5c_570bed30-5e16-475c-9cc4-4e5af148c2e5/global-pull-secret-syncer/0.log" Jul 07 19:09:05.829591 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:05.829545 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-w4rcb_330e5b54-ff72-4853-b885-9a3327275f99/konnectivity-agent/0.log" Jul 07 19:09:05.941257 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:05.941233 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-81.ec2.internal_c7bff29c072ef9b7b9d4f7fce9f2d5cc/haproxy/0.log" Jul 07 19:09:12.670267 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:12.670154 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-m8fgs_64e169d3-cc4a-46a0-bfe6-1fb28c8deeae/cluster-monitoring-operator/0.log" Jul 07 19:09:12.699215 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:12.699160 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-rcvxn_7566d1fb-829e-4376-a8d6-fc2239d8e32c/kube-state-metrics/0.log" Jul 07 19:09:12.722654 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:12.722609 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-rcvxn_7566d1fb-829e-4376-a8d6-fc2239d8e32c/kube-rbac-proxy-main/0.log" Jul 07 19:09:12.748919 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:12.748884 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-rcvxn_7566d1fb-829e-4376-a8d6-fc2239d8e32c/kube-rbac-proxy-self/0.log" Jul 07 19:09:12.839157 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:12.839111 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4s89j_2e5e2610-e42a-4689-990c-d681773d0d1e/node-exporter/0.log" Jul 07 19:09:12.872888 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:12.872843 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4s89j_2e5e2610-e42a-4689-990c-d681773d0d1e/kube-rbac-proxy/0.log" Jul 07 19:09:12.909861 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:12.909834 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4s89j_2e5e2610-e42a-4689-990c-d681773d0d1e/init-textfile/0.log" Jul 07 19:09:13.150187 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:13.150159 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-nkx47_7e8311d0-5139-405b-9fcb-709fceb273db/kube-rbac-proxy-main/0.log" Jul 07 19:09:13.176234 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:13.176149 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-nkx47_7e8311d0-5139-405b-9fcb-709fceb273db/kube-rbac-proxy-self/0.log" Jul 07 19:09:13.210407 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:13.210358 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-nkx47_7e8311d0-5139-405b-9fcb-709fceb273db/openshift-state-metrics/0.log" Jul 07 19:09:13.254550 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:13.254510 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_90e33b63-2861-47a3-ad96-e6aff8096bda/prometheus/0.log" Jul 07 19:09:13.281211 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:13.281148 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_90e33b63-2861-47a3-ad96-e6aff8096bda/config-reloader/0.log" Jul 07 19:09:13.312332 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:13.312300 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_90e33b63-2861-47a3-ad96-e6aff8096bda/thanos-sidecar/0.log" Jul 07 19:09:13.341570 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:13.341540 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_90e33b63-2861-47a3-ad96-e6aff8096bda/kube-rbac-proxy-web/0.log" Jul 07 19:09:13.370175 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:13.370145 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_90e33b63-2861-47a3-ad96-e6aff8096bda/kube-rbac-proxy/0.log" Jul 07 19:09:13.395650 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:13.395623 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_90e33b63-2861-47a3-ad96-e6aff8096bda/kube-rbac-proxy-thanos/0.log" Jul 07 19:09:13.423083 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:13.423048 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_90e33b63-2861-47a3-ad96-e6aff8096bda/init-config-reloader/0.log" Jul 07 19:09:13.452896 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:13.452862 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-hfvdd_05c43c32-65c0-402e-80b6-d0da89b9a7a0/prometheus-operator/0.log" Jul 07 19:09:13.479682 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:13.479611 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-hfvdd_05c43c32-65c0-402e-80b6-d0da89b9a7a0/kube-rbac-proxy/0.log" Jul 07 19:09:13.514171 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:13.514140 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-bsgkt_a70abc33-74af-46f8-8f77-e1f0b3065270/prometheus-operator-admission-webhook/0.log" Jul 07 19:09:13.644526 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:13.644491 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8488ddff78-wthfd_f3010063-bc1a-4d37-95fb-c27e7d9720e7/thanos-query/0.log" Jul 07 19:09:13.668415 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:13.668384 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8488ddff78-wthfd_f3010063-bc1a-4d37-95fb-c27e7d9720e7/kube-rbac-proxy-web/0.log" Jul 07 19:09:13.690627 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:13.690598 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8488ddff78-wthfd_f3010063-bc1a-4d37-95fb-c27e7d9720e7/kube-rbac-proxy/0.log" Jul 07 19:09:13.716602 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:13.716572 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8488ddff78-wthfd_f3010063-bc1a-4d37-95fb-c27e7d9720e7/prom-label-proxy/0.log" Jul 07 19:09:13.746014 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:13.745902 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8488ddff78-wthfd_f3010063-bc1a-4d37-95fb-c27e7d9720e7/kube-rbac-proxy-rules/0.log" Jul 07 19:09:13.771584 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:13.771532 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8488ddff78-wthfd_f3010063-bc1a-4d37-95fb-c27e7d9720e7/kube-rbac-proxy-metrics/0.log" Jul 07 19:09:15.040787 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:15.040762 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-cxwq7_9250496d-2e44-489b-b3c0-b6c3121077a3/networking-console-plugin/0.log" Jul 07 19:09:15.678917 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:15.678884 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf"] Jul 07 19:09:15.686756 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:15.686731 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" Jul 07 19:09:15.697472 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:15.696860 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf"] Jul 07 19:09:15.862907 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:15.862862 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2321cfbb-92c3-43cd-88a3-a7aca66d002a-podres\") pod \"perf-node-gather-daemonset-twjgf\" (UID: \"2321cfbb-92c3-43cd-88a3-a7aca66d002a\") " pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" Jul 07 19:09:15.863101 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:15.862948 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2321cfbb-92c3-43cd-88a3-a7aca66d002a-proc\") pod \"perf-node-gather-daemonset-twjgf\" (UID: \"2321cfbb-92c3-43cd-88a3-a7aca66d002a\") " pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" Jul 07 19:09:15.863101 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:15.863016 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2321cfbb-92c3-43cd-88a3-a7aca66d002a-lib-modules\") pod \"perf-node-gather-daemonset-twjgf\" (UID: \"2321cfbb-92c3-43cd-88a3-a7aca66d002a\") " pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" Jul 07 19:09:15.863101 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:15.863079 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2321cfbb-92c3-43cd-88a3-a7aca66d002a-sys\") pod \"perf-node-gather-daemonset-twjgf\" (UID: \"2321cfbb-92c3-43cd-88a3-a7aca66d002a\") " pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" Jul 07 19:09:15.863316 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:15.863114 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m74b6\" (UniqueName: \"kubernetes.io/projected/2321cfbb-92c3-43cd-88a3-a7aca66d002a-kube-api-access-m74b6\") pod \"perf-node-gather-daemonset-twjgf\" (UID: \"2321cfbb-92c3-43cd-88a3-a7aca66d002a\") " pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" Jul 07 19:09:15.963898 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:15.963868 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2321cfbb-92c3-43cd-88a3-a7aca66d002a-sys\") pod \"perf-node-gather-daemonset-twjgf\" (UID: \"2321cfbb-92c3-43cd-88a3-a7aca66d002a\") " pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" Jul 07 19:09:15.964070 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:15.963908 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m74b6\" (UniqueName: \"kubernetes.io/projected/2321cfbb-92c3-43cd-88a3-a7aca66d002a-kube-api-access-m74b6\") pod \"perf-node-gather-daemonset-twjgf\" (UID: \"2321cfbb-92c3-43cd-88a3-a7aca66d002a\") " pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" Jul 07 19:09:15.964070 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:15.963951 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2321cfbb-92c3-43cd-88a3-a7aca66d002a-podres\") pod \"perf-node-gather-daemonset-twjgf\" (UID: \"2321cfbb-92c3-43cd-88a3-a7aca66d002a\") " pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" Jul 07 19:09:15.964070 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:15.963995 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2321cfbb-92c3-43cd-88a3-a7aca66d002a-sys\") pod \"perf-node-gather-daemonset-twjgf\" (UID: \"2321cfbb-92c3-43cd-88a3-a7aca66d002a\") " pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" Jul 07 19:09:15.964070 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:15.964003 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2321cfbb-92c3-43cd-88a3-a7aca66d002a-proc\") pod \"perf-node-gather-daemonset-twjgf\" (UID: \"2321cfbb-92c3-43cd-88a3-a7aca66d002a\") " pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" Jul 07 19:09:15.964070 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:15.964054 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2321cfbb-92c3-43cd-88a3-a7aca66d002a-lib-modules\") pod \"perf-node-gather-daemonset-twjgf\" (UID: \"2321cfbb-92c3-43cd-88a3-a7aca66d002a\") " pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" Jul 07 19:09:15.964268 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:15.964149 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/2321cfbb-92c3-43cd-88a3-a7aca66d002a-proc\") pod \"perf-node-gather-daemonset-twjgf\" (UID: \"2321cfbb-92c3-43cd-88a3-a7aca66d002a\") " pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" Jul 07 19:09:15.964268 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:15.964183 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/2321cfbb-92c3-43cd-88a3-a7aca66d002a-lib-modules\") pod \"perf-node-gather-daemonset-twjgf\" (UID: \"2321cfbb-92c3-43cd-88a3-a7aca66d002a\") " pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" Jul 07 19:09:15.964341 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:15.964265 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/2321cfbb-92c3-43cd-88a3-a7aca66d002a-podres\") pod \"perf-node-gather-daemonset-twjgf\" (UID: \"2321cfbb-92c3-43cd-88a3-a7aca66d002a\") " pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" Jul 07 19:09:15.976602 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:15.976570 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m74b6\" (UniqueName: \"kubernetes.io/projected/2321cfbb-92c3-43cd-88a3-a7aca66d002a-kube-api-access-m74b6\") pod \"perf-node-gather-daemonset-twjgf\" (UID: \"2321cfbb-92c3-43cd-88a3-a7aca66d002a\") " pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" Jul 07 19:09:16.000633 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:16.000604 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" Jul 07 19:09:16.187158 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:16.187126 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf"] Jul 07 19:09:16.269323 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:16.269303 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-9b4bdcc46-7nw9b_1ec7a05c-4b58-435b-bf38-c5134e09fae7/console/0.log" Jul 07 19:09:16.909300 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:16.909259 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" event={"ID":"2321cfbb-92c3-43cd-88a3-a7aca66d002a","Type":"ContainerStarted","Data":"6e6276536bb4739e1ad9cc372b4980fd7210444a3bb79c78bbe85daea2e22001"} Jul 07 19:09:16.909499 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:16.909328 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" event={"ID":"2321cfbb-92c3-43cd-88a3-a7aca66d002a","Type":"ContainerStarted","Data":"edeb4618bb1ce0d63db77f273a2c8da9d462316afa217dd47da561561f9eab4e"} Jul 07 19:09:16.910173 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:16.910143 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" Jul 07 19:09:16.928609 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:16.928549 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" podStartSLOduration=1.9285350380000001 podStartE2EDuration="1.928535038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 19:09:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 19:09:16.926963485 +0000 UTC m=+1168.896599505" watchObservedRunningTime="2026-07-07 19:09:16.928535038 +0000 UTC m=+1168.898171054" Jul 07 19:09:18.017952 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:18.017926 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b57q9_fa1690c9-bef6-43d0-9aff-2359f4dfc39f/dns/0.log" Jul 07 19:09:18.046278 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:18.046244 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b57q9_fa1690c9-bef6-43d0-9aff-2359f4dfc39f/kube-rbac-proxy/0.log" Jul 07 19:09:18.195651 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:18.195617 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8bhnt_7dcc43c9-3758-47a3-aa8d-278166547dad/dns-node-resolver/0.log" Jul 07 19:09:18.855351 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:18.855323 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qkhdg_8821d59a-1e1c-43d8-80d8-3513deb7478b/node-ca/0.log" Jul 07 19:09:20.003080 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:20.003052 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85b6b9f996-bdq8n_c93fe07d-9362-4713-9dee-e9217aa394e3/kube-auth-proxy/0.log" Jul 07 19:09:20.681963 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:20.681939 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bphf7_cf6f4b56-2567-469f-a0d4-bc586ab3c6bf/serve-healthcheck-canary/0.log" Jul 07 19:09:21.219762 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:21.219732 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-trfsb_1ccc5d0f-1bc6-4866-b99a-82252b234d0f/insights-operator/1.log" Jul 07 19:09:21.219762 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:21.219732 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-trfsb_1ccc5d0f-1bc6-4866-b99a-82252b234d0f/insights-operator/0.log" Jul 07 19:09:21.391811 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:21.391781 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lksfn_5a4777b9-2b1e-4987-9073-426919e6919c/kube-rbac-proxy/0.log" Jul 07 19:09:21.420147 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:21.420109 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lksfn_5a4777b9-2b1e-4987-9073-426919e6919c/exporter/0.log" Jul 07 19:09:21.446533 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:21.446507 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lksfn_5a4777b9-2b1e-4987-9073-426919e6919c/extractor/0.log" Jul 07 19:09:23.667460 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:23.667425 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-645cb998cd-klzwt_4e026420-a6eb-475e-8f43-4fd02c884819/manager/0.log" Jul 07 19:09:23.783660 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:23.783626 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr_5f7c7b22-3588-4137-ba9f-2dd8d8889ae6/manager/0.log" Jul 07 19:09:23.804550 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:23.804520 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d6ffc4cd5-5w9gr_5f7c7b22-3588-4137-ba9f-2dd8d8889ae6/copy-manifests/0.log" Jul 07 19:09:23.933819 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:23.933744 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zqqk9/perf-node-gather-daemonset-twjgf" Jul 07 19:09:23.970526 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:23.970496 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-n4dv8_4946fcd5-d821-4a02-bf11-23b7658109f0/postgres/0.log" Jul 07 19:09:25.183022 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:25.182980 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6d5b9d7dd9-kbtbt_db38a546-4d03-4882-942c-3da5e96a7400/manager/0.log" Jul 07 19:09:25.395069 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:25.395040 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85b6b9f996-bdq8n_c93fe07d-9362-4713-9dee-e9217aa394e3/kube-auth-proxy/0.log" Jul 07 19:09:26.750176 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:26.750101 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85b6b9f996-bdq8n_c93fe07d-9362-4713-9dee-e9217aa394e3/kube-auth-proxy/0.log" Jul 07 19:09:30.924852 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:30.924819 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-d4mz6_7049298b-743d-4673-a84f-4e2465166aa3/kube-storage-version-migrator-operator/1.log" Jul 07 19:09:30.926058 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:30.926031 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-d4mz6_7049298b-743d-4673-a84f-4e2465166aa3/kube-storage-version-migrator-operator/0.log" Jul 07 19:09:32.140741 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:32.140713 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6cb6g_b8a9c668-2d9f-4c07-9b0e-e9e57c20a796/kube-multus-additional-cni-plugins/0.log" Jul 07 19:09:32.176799 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:32.176768 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6cb6g_b8a9c668-2d9f-4c07-9b0e-e9e57c20a796/egress-router-binary-copy/0.log" Jul 07 19:09:32.199078 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:32.199041 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6cb6g_b8a9c668-2d9f-4c07-9b0e-e9e57c20a796/cni-plugins/0.log" Jul 07 19:09:32.225316 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:32.225288 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6cb6g_b8a9c668-2d9f-4c07-9b0e-e9e57c20a796/bond-cni-plugin/0.log" Jul 07 19:09:32.248206 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:32.248168 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6cb6g_b8a9c668-2d9f-4c07-9b0e-e9e57c20a796/routeoverride-cni/0.log" Jul 07 19:09:32.273039 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:32.273007 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6cb6g_b8a9c668-2d9f-4c07-9b0e-e9e57c20a796/whereabouts-cni-bincopy/0.log" Jul 07 19:09:32.296827 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:32.296794 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6cb6g_b8a9c668-2d9f-4c07-9b0e-e9e57c20a796/whereabouts-cni/0.log" Jul 07 19:09:32.509443 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:32.509413 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-kk8zs_0d95ab7d-5d1d-4026-b705-72ea50c1ad51/kube-multus/0.log" Jul 07 19:09:32.613243 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:32.613217 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ccjd5_536dbfa9-2261-4932-a922-e11a2a764168/network-metrics-daemon/0.log" Jul 07 19:09:32.661159 ip-10-0-135-81 kubenswrapper[2564]: I0707 19:09:32.661137 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ccjd5_536dbfa9-2261-4932-a922-e11a2a764168/kube-rbac-proxy/0.log"