Jul 02 13:41:32.523174 ip-10-0-141-112 systemd[1]: Starting Kubernetes Kubelet... Jul 02 13:41:34.202333 ip-10-0-141-112 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 13:41:34.202333 ip-10-0-141-112 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 02 13:41:34.202333 ip-10-0-141-112 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 13:41:34.202333 ip-10-0-141-112 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 02 13:41:34.202333 ip-10-0-141-112 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 13:41:34.204321 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.204188 2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 02 13:41:34.208764 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208740 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 13:41:34.208764 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208765 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 13:41:34.208842 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208770 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 13:41:34.208842 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208774 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 13:41:34.208842 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208777 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 13:41:34.208842 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208781 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 13:41:34.208842 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208784 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 13:41:34.208842 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208787 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 13:41:34.208842 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208790 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 13:41:34.208842 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208793 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 13:41:34.208842 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208796 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 13:41:34.208842 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208798 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 13:41:34.208842 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208801 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 13:41:34.208842 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208803 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 13:41:34.208842 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208806 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 13:41:34.208842 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208809 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 13:41:34.208842 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208811 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 13:41:34.208842 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208814 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 13:41:34.208842 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208816 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 13:41:34.208842 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208819 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 13:41:34.208842 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208821 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 13:41:34.208842 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208841 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 13:41:34.209335 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208845 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 13:41:34.209335 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208848 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 13:41:34.209335 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208851 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 13:41:34.209335 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208854 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 13:41:34.209335 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208857 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 13:41:34.209335 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208860 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 13:41:34.209335 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208863 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 13:41:34.209335 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208865 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 13:41:34.209335 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208868 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 13:41:34.209335 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208870 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 13:41:34.209335 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208873 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 13:41:34.209335 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208876 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 13:41:34.209335 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208878 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 13:41:34.209335 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208881 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 13:41:34.209335 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208883 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 13:41:34.209335 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208886 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 13:41:34.209335 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208888 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 13:41:34.209335 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208891 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 13:41:34.209335 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208894 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 13:41:34.209335 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208897 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 13:41:34.209818 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208900 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 13:41:34.209818 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208902 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 13:41:34.209818 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208905 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 13:41:34.209818 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208908 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 13:41:34.209818 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208910 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 13:41:34.209818 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208913 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 13:41:34.209818 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208916 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 13:41:34.209818 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208918 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 13:41:34.209818 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208921 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 13:41:34.209818 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208924 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 13:41:34.209818 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208929 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 13:41:34.209818 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208931 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 13:41:34.209818 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208934 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 13:41:34.209818 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208938 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 13:41:34.209818 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208941 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 13:41:34.209818 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208945 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 13:41:34.209818 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208949 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 13:41:34.209818 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208952 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 13:41:34.209818 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208955 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 13:41:34.209818 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208957 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 13:41:34.210401 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208960 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 13:41:34.210401 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208963 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 13:41:34.210401 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208966 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 13:41:34.210401 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208984 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 13:41:34.210401 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208990 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 13:41:34.210401 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208993 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 13:41:34.210401 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208996 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 13:41:34.210401 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.208999 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 13:41:34.210401 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209001 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 13:41:34.210401 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209005 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 13:41:34.210401 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209008 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 13:41:34.210401 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209010 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 13:41:34.210401 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209013 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 13:41:34.210401 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209016 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 13:41:34.210401 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209019 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 02 13:41:34.210401 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209021 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 13:41:34.210401 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209025 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 13:41:34.210401 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209027 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 13:41:34.210401 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209030 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 13:41:34.210401 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209033 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 13:41:34.210882 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209036 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 13:41:34.210882 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209038 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 13:41:34.210882 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209041 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 13:41:34.210882 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209043 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 13:41:34.210882 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209046 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 13:41:34.210882 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209049 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 13:41:34.210882 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209053 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 13:41:34.210882 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209055 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 13:41:34.210882 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209058 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 13:41:34.210882 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209061 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 13:41:34.210882 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209063 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 13:41:34.210882 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209066 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 13:41:34.210882 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209068 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 13:41:34.210882 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209071 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 13:41:34.210882 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209073 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 13:41:34.210882 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209075 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 13:41:34.210882 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209079 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 13:41:34.210882 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209812 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 13:41:34.210882 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209819 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 13:41:34.211392 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209822 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 13:41:34.211392 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209825 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 13:41:34.211392 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209828 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 13:41:34.211392 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209831 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 13:41:34.211392 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209834 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 13:41:34.211392 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209837 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 13:41:34.211392 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209840 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 13:41:34.211392 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209842 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 13:41:34.211392 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209845 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 13:41:34.211392 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209848 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 13:41:34.211392 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209850 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 13:41:34.211392 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209853 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 13:41:34.211392 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209856 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 13:41:34.211392 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209858 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 13:41:34.211392 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209861 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 13:41:34.211392 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209863 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 13:41:34.211392 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209866 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 13:41:34.211392 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209870 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 13:41:34.211392 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209873 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 13:41:34.211392 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209876 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 13:41:34.211890 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209881 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 13:41:34.211890 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209885 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 13:41:34.211890 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209889 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 13:41:34.211890 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209892 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 13:41:34.211890 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209895 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 13:41:34.211890 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209899 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 13:41:34.211890 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209901 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 13:41:34.211890 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209904 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 13:41:34.211890 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209907 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 13:41:34.211890 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209910 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 13:41:34.211890 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209912 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 13:41:34.211890 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209915 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 13:41:34.211890 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209918 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 13:41:34.211890 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209920 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 13:41:34.211890 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209923 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 13:41:34.211890 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209926 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 13:41:34.211890 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209929 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 13:41:34.211890 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209932 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 13:41:34.211890 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209934 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 13:41:34.211890 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209937 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 13:41:34.212387 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209939 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 13:41:34.212387 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209942 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 13:41:34.212387 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209944 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 13:41:34.212387 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209947 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 13:41:34.212387 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209949 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 13:41:34.212387 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209951 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 13:41:34.212387 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209955 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 13:41:34.212387 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209959 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 13:41:34.212387 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209961 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 13:41:34.212387 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209964 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 13:41:34.212387 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209968 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 13:41:34.212387 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209989 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 13:41:34.212387 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209992 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 13:41:34.212387 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209995 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 13:41:34.212387 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.209998 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 13:41:34.212387 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210002 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 13:41:34.212387 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210004 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 13:41:34.212387 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210007 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 13:41:34.212387 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210009 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 13:41:34.212387 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210012 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 13:41:34.212875 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210015 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 13:41:34.212875 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210017 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 13:41:34.212875 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210020 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 13:41:34.212875 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210022 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 13:41:34.212875 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210025 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 13:41:34.212875 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210028 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 13:41:34.212875 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210031 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 13:41:34.212875 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210033 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 13:41:34.212875 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210036 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 13:41:34.212875 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210040 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 13:41:34.212875 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210042 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 13:41:34.212875 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210045 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 13:41:34.212875 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210047 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 13:41:34.212875 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210050 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 13:41:34.212875 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210053 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 13:41:34.212875 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210055 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 13:41:34.212875 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210059 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 13:41:34.212875 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210061 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 13:41:34.212875 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210064 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 13:41:34.213382 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210066 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 13:41:34.213382 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210069 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 13:41:34.213382 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210071 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 13:41:34.213382 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210075 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 13:41:34.213382 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210077 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 13:41:34.213382 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210080 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 13:41:34.213382 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210083 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 13:41:34.213382 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210085 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 13:41:34.213382 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210089 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 13:41:34.213382 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210093 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 13:41:34.213382 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210096 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 13:41:34.213382 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210100 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 13:41:34.213382 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210102 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 13:41:34.213382 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210105 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 13:41:34.213382 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210108 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 13:41:34.213382 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210111 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 02 13:41:34.213382 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210114 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 13:41:34.213382 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210117 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 13:41:34.213382 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210519 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 13:41:34.213853 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210524 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 13:41:34.213853 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210527 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 13:41:34.213853 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210530 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 13:41:34.213853 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210533 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 13:41:34.213853 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210536 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 13:41:34.213853 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210538 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 13:41:34.213853 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210541 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 13:41:34.213853 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210543 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 13:41:34.213853 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210546 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 13:41:34.213853 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210549 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 13:41:34.213853 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210551 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 13:41:34.213853 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210554 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 13:41:34.213853 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210557 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 13:41:34.213853 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210560 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 13:41:34.213853 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210564 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 13:41:34.213853 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210569 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 13:41:34.213853 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210573 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 13:41:34.213853 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210576 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 13:41:34.213853 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210579 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 13:41:34.213853 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210581 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 13:41:34.214357 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210584 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 13:41:34.214357 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210586 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 13:41:34.214357 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210589 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 13:41:34.214357 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210591 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 13:41:34.214357 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210594 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 13:41:34.214357 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210596 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 13:41:34.214357 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210599 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 13:41:34.214357 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210601 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 13:41:34.214357 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210604 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 13:41:34.214357 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210606 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 02 13:41:34.214357 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210609 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 13:41:34.214357 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210612 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 13:41:34.214357 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210614 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 13:41:34.214357 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210616 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 13:41:34.214357 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210619 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 13:41:34.214357 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210622 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 13:41:34.214357 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210625 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 13:41:34.214357 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210628 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 13:41:34.214357 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210630 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 13:41:34.214357 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210633 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 13:41:34.214879 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210635 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 13:41:34.214879 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210638 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 13:41:34.214879 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210640 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 13:41:34.214879 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210643 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 13:41:34.214879 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210646 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 13:41:34.214879 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210650 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 13:41:34.214879 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210652 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 13:41:34.214879 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210658 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 13:41:34.214879 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210660 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 13:41:34.214879 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210663 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 13:41:34.214879 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210665 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 13:41:34.214879 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210668 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 13:41:34.214879 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210670 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 13:41:34.214879 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210673 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 13:41:34.214879 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210675 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 13:41:34.214879 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210678 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 13:41:34.214879 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210680 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 13:41:34.214879 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210683 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 13:41:34.214879 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210685 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 13:41:34.215373 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210688 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 13:41:34.215373 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210692 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 13:41:34.215373 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210694 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 13:41:34.215373 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210696 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 13:41:34.215373 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210700 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 13:41:34.215373 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210702 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 13:41:34.215373 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210707 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 13:41:34.215373 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210710 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 13:41:34.215373 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210714 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 13:41:34.215373 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210717 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 13:41:34.215373 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210721 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 13:41:34.215373 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210724 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 13:41:34.215373 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210727 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 13:41:34.215373 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210729 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 13:41:34.215373 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210732 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 13:41:34.215373 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210735 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 13:41:34.215373 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210737 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 13:41:34.215373 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210740 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 13:41:34.215373 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210742 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 13:41:34.215852 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210745 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 13:41:34.215852 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210749 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 13:41:34.215852 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210751 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 13:41:34.215852 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210754 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 13:41:34.215852 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210756 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 13:41:34.215852 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210759 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 13:41:34.215852 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210761 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 13:41:34.215852 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210764 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 13:41:34.215852 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210767 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 13:41:34.215852 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210770 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 13:41:34.215852 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210772 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 13:41:34.215852 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210775 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 13:41:34.215852 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210777 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 13:41:34.215852 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210780 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 13:41:34.215852 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210783 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 13:41:34.215852 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210785 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 13:41:34.215852 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210788 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 13:41:34.215852 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210792 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 13:41:34.215852 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210795 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.210797 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210878 2561 flags.go:64] FLAG: --address="0.0.0.0" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210886 2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210893 2561 flags.go:64] FLAG: --anonymous-auth="true" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210897 2561 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210903 2561 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210907 2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210911 2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210916 2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210919 2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210922 2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210926 2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210929 2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210932 2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210937 2561 flags.go:64] FLAG: --cgroup-root="" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210940 2561 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210943 2561 flags.go:64] FLAG: --client-ca-file="" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210947 2561 flags.go:64] FLAG: --cloud-provider="external" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210950 2561 flags.go:64] FLAG: --cluster-dns="[]" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210954 2561 flags.go:64] FLAG: --cluster-domain="" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210957 2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210960 2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210963 2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 02 13:41:34.216354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210968 2561 flags.go:64] FLAG: --container-log-max-files="5" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210989 2561 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210994 2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.210998 2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211002 2561 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211006 2561 flags.go:64] FLAG: --contention-profiling="false" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211009 2561 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211014 2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211018 2561 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211021 2561 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211026 2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211029 2561 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211032 2561 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211035 2561 flags.go:64] FLAG: --enable-load-reader="false" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211038 2561 flags.go:64] FLAG: --enable-server="true" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211042 2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211046 2561 flags.go:64] FLAG: --event-burst="100" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211050 2561 flags.go:64] FLAG: --event-qps="50" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211053 2561 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211056 2561 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211059 2561 flags.go:64] FLAG: --eviction-hard="" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211063 2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211066 2561 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211070 2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211075 2561 flags.go:64] FLAG: --eviction-soft="" Jul 02 13:41:34.216931 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211077 2561 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211080 2561 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211084 2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211087 2561 flags.go:64] FLAG: --experimental-mounter-path="" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211090 2561 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211093 2561 flags.go:64] FLAG: --fail-swap-on="true" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211096 2561 flags.go:64] FLAG: --feature-gates="" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211100 2561 flags.go:64] FLAG: --file-check-frequency="20s" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211103 2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211106 2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211109 2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211112 2561 flags.go:64] FLAG: --healthz-port="10248" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211115 2561 flags.go:64] FLAG: --help="false" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211118 2561 flags.go:64] FLAG: --hostname-override="ip-10-0-141-112.ec2.internal" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211123 2561 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211126 2561 flags.go:64] FLAG: --http-check-frequency="20s" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211129 2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211132 2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211135 2561 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211138 2561 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211141 2561 flags.go:64] FLAG: --image-service-endpoint="" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211144 2561 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211146 2561 flags.go:64] FLAG: --kube-api-burst="100" Jul 02 13:41:34.217545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211150 2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211153 2561 flags.go:64] FLAG: --kube-api-qps="50" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211156 2561 flags.go:64] FLAG: --kube-reserved="" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211159 2561 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211161 2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211164 2561 flags.go:64] FLAG: --kubelet-cgroups="" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211167 2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211171 2561 flags.go:64] FLAG: --lock-file="" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211174 2561 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211177 2561 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211180 2561 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211185 2561 flags.go:64] FLAG: --log-json-split-stream="false" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211188 2561 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211199 2561 flags.go:64] FLAG: --log-text-split-stream="false" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211202 2561 flags.go:64] FLAG: --logging-format="text" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211205 2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211208 2561 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211211 2561 flags.go:64] FLAG: --manifest-url="" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211214 2561 flags.go:64] FLAG: --manifest-url-header="" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211224 2561 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211227 2561 flags.go:64] FLAG: --max-open-files="1000000" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211231 2561 flags.go:64] FLAG: --max-pods="110" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211235 2561 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211239 2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211242 2561 flags.go:64] FLAG: --memory-manager-policy="None" Jul 02 13:41:34.218158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211245 2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211248 2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211251 2561 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211254 2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211262 2561 flags.go:64] FLAG: --node-status-max-images="50" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211265 2561 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211269 2561 flags.go:64] FLAG: --oom-score-adj="-999" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211271 2561 flags.go:64] FLAG: --pod-cidr="" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211275 2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211282 2561 flags.go:64] FLAG: --pod-manifest-path="" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211285 2561 flags.go:64] FLAG: --pod-max-pids="-1" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211288 2561 flags.go:64] FLAG: --pods-per-core="0" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211291 2561 flags.go:64] FLAG: --port="10250" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211294 2561 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211297 2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fa8cc3517dec379f" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211301 2561 flags.go:64] FLAG: --qos-reserved="" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211304 2561 flags.go:64] FLAG: --read-only-port="10255" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211307 2561 flags.go:64] FLAG: --register-node="true" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211311 2561 flags.go:64] FLAG: --register-with-taints="" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211315 2561 flags.go:64] FLAG: --registry-burst="10" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211317 2561 flags.go:64] FLAG: --registry-qps="5" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211320 2561 flags.go:64] FLAG: --reserved-cpus="" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211323 2561 flags.go:64] FLAG: --reserved-memory="" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211327 2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211330 2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 02 13:41:34.218786 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211333 2561 flags.go:64] FLAG: --rotate-certificates="false" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211336 2561 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211339 2561 flags.go:64] FLAG: --runonce="false" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211342 2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211345 2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211350 2561 flags.go:64] FLAG: --seccomp-default="false" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211353 2561 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211356 2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211359 2561 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211362 2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211365 2561 flags.go:64] FLAG: --storage-driver-password="root" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211368 2561 flags.go:64] FLAG: --storage-driver-secure="false" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211371 2561 flags.go:64] FLAG: --storage-driver-table="stats" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211374 2561 flags.go:64] FLAG: --storage-driver-user="root" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211377 2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211380 2561 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211384 2561 flags.go:64] FLAG: --system-cgroups="" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211387 2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211392 2561 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211395 2561 flags.go:64] FLAG: --tls-cert-file="" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211398 2561 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211402 2561 flags.go:64] FLAG: --tls-min-version="" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211406 2561 flags.go:64] FLAG: --tls-private-key-file="" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211409 2561 flags.go:64] FLAG: --topology-manager-policy="none" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211412 2561 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 02 13:41:34.219460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211415 2561 flags.go:64] FLAG: --topology-manager-scope="container" Jul 02 13:41:34.220165 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211418 2561 flags.go:64] FLAG: --v="2" Jul 02 13:41:34.220165 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211422 2561 flags.go:64] FLAG: --version="false" Jul 02 13:41:34.220165 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211426 2561 flags.go:64] FLAG: --vmodule="" Jul 02 13:41:34.220165 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211430 2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 02 13:41:34.220165 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.211433 2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 02 13:41:34.220165 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211570 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 13:41:34.220165 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211575 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 13:41:34.220165 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211579 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 13:41:34.220165 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211583 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 13:41:34.220165 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211587 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 13:41:34.220165 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211592 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 13:41:34.220165 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211599 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 13:41:34.220165 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211603 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 13:41:34.220165 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211607 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 13:41:34.220165 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211611 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 13:41:34.220165 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211615 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 13:41:34.220165 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211619 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 13:41:34.220165 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211623 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 13:41:34.220165 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211627 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 13:41:34.220165 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211631 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 13:41:34.220165 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211634 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 13:41:34.220678 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211638 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 13:41:34.220678 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211641 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 13:41:34.220678 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211652 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 13:41:34.220678 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211656 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 13:41:34.220678 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211662 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 13:41:34.220678 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211666 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 13:41:34.220678 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211671 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 13:41:34.220678 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211675 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 13:41:34.220678 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211679 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 13:41:34.220678 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211683 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 02 13:41:34.220678 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211687 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 13:41:34.220678 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211690 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 13:41:34.220678 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211695 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 13:41:34.220678 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211698 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 13:41:34.220678 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211702 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 13:41:34.220678 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211705 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 13:41:34.220678 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211709 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 13:41:34.220678 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211712 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 13:41:34.220678 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211716 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 13:41:34.220678 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211719 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 13:41:34.221177 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211723 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 13:41:34.221177 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211727 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 13:41:34.221177 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211731 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 13:41:34.221177 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211735 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 13:41:34.221177 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211739 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 13:41:34.221177 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211743 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 13:41:34.221177 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211747 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 13:41:34.221177 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211751 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 13:41:34.221177 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211755 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 13:41:34.221177 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211758 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 13:41:34.221177 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211762 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 13:41:34.221177 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211766 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 13:41:34.221177 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211769 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 13:41:34.221177 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211773 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 13:41:34.221177 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211777 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 13:41:34.221177 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211781 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 13:41:34.221177 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211785 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 13:41:34.221177 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211788 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 13:41:34.221177 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211792 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 13:41:34.221177 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211796 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 13:41:34.221663 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211800 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 13:41:34.221663 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211804 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 13:41:34.221663 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211808 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 13:41:34.221663 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211812 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 13:41:34.221663 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211816 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 13:41:34.221663 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211821 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 13:41:34.221663 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211826 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 13:41:34.221663 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211831 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 13:41:34.221663 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211834 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 13:41:34.221663 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211838 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 13:41:34.221663 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211842 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 13:41:34.221663 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211846 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 13:41:34.221663 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211850 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 13:41:34.221663 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211865 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 13:41:34.221663 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211870 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 13:41:34.221663 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211873 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 13:41:34.221663 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211875 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 13:41:34.221663 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211878 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 13:41:34.221663 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211880 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 13:41:34.222144 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211883 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 13:41:34.222144 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211886 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 13:41:34.222144 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211889 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 13:41:34.222144 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211891 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 13:41:34.222144 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211894 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 13:41:34.222144 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211897 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 13:41:34.222144 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211900 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 13:41:34.222144 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211902 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 13:41:34.222144 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211905 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 13:41:34.222144 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211910 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 13:41:34.222144 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211912 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 13:41:34.222144 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211915 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 13:41:34.222144 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211918 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 13:41:34.222144 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211921 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 13:41:34.222144 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211923 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 13:41:34.222144 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211925 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 13:41:34.222144 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211928 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 13:41:34.222144 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211931 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 13:41:34.222144 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211934 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 13:41:34.222144 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211937 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 13:41:34.222626 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211939 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 13:41:34.222626 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211941 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 13:41:34.222626 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211944 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 13:41:34.222626 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.211947 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 13:41:34.222626 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.212629 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 13:41:34.222626 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.220010 2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 02 13:41:34.222626 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.220032 2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 02 13:41:34.222626 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.221270 2561 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 02 13:41:34.222626 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.221283 2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 02 13:41:34.222626 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221357 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 13:41:34.222626 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221364 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 13:41:34.222626 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221367 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 13:41:34.222626 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221371 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 13:41:34.222626 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221374 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 13:41:34.222991 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221377 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 13:41:34.222991 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221380 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 13:41:34.222991 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221383 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 13:41:34.222991 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221385 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 13:41:34.222991 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221388 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 13:41:34.222991 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221391 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 13:41:34.222991 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221393 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 13:41:34.222991 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221396 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 13:41:34.222991 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221398 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 13:41:34.222991 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221401 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 13:41:34.222991 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221404 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 13:41:34.222991 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221407 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 13:41:34.222991 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221409 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 13:41:34.222991 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221412 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 13:41:34.222991 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221414 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 13:41:34.222991 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221417 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 13:41:34.222991 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221420 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 13:41:34.222991 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221422 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 13:41:34.222991 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221425 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 13:41:34.223468 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221428 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 13:41:34.223468 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221430 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 13:41:34.223468 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221433 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 13:41:34.223468 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221436 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 13:41:34.223468 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221439 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 13:41:34.223468 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221449 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 13:41:34.223468 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221452 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 13:41:34.223468 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221455 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 13:41:34.223468 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221457 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 13:41:34.223468 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221460 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 13:41:34.223468 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221463 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 13:41:34.223468 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221466 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 13:41:34.223468 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221468 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 13:41:34.223468 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221471 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 13:41:34.223468 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221473 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 13:41:34.223468 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221476 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 13:41:34.223468 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221479 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 13:41:34.223468 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221481 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 13:41:34.223468 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221484 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 13:41:34.223468 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221487 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 13:41:34.223994 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221489 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 13:41:34.223994 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221492 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 13:41:34.223994 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221495 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 13:41:34.223994 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221497 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 13:41:34.223994 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221500 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 13:41:34.223994 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221502 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 13:41:34.223994 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221505 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 13:41:34.223994 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221508 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 13:41:34.223994 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221510 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 13:41:34.223994 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221513 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 13:41:34.223994 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221515 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 13:41:34.223994 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221518 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 13:41:34.223994 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221521 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 13:41:34.223994 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221523 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 13:41:34.223994 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221526 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 13:41:34.223994 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221529 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 13:41:34.223994 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221532 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 13:41:34.223994 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221535 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 13:41:34.223994 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221544 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 13:41:34.223994 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221547 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 13:41:34.224781 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221549 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 13:41:34.224781 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221552 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 13:41:34.224781 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221555 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 13:41:34.224781 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221557 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 13:41:34.224781 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221560 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 13:41:34.224781 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221562 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 13:41:34.224781 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221564 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 13:41:34.224781 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221567 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 13:41:34.224781 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221569 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 13:41:34.224781 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221572 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 13:41:34.224781 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221576 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 13:41:34.224781 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221580 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 13:41:34.224781 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221584 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 13:41:34.224781 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221587 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 13:41:34.224781 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221589 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 13:41:34.224781 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221592 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 13:41:34.224781 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221594 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 13:41:34.224781 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221597 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 13:41:34.224781 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221601 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 13:41:34.225534 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221605 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 13:41:34.225534 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221609 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 13:41:34.225534 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221612 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 13:41:34.225534 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221615 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 02 13:41:34.225534 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221618 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 13:41:34.225534 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221621 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 13:41:34.225534 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221624 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 13:41:34.225534 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221627 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 13:41:34.225534 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221630 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 13:41:34.225534 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221633 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 13:41:34.225534 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221636 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 13:41:34.225534 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221639 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 13:41:34.225534 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221648 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 13:41:34.225534 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221650 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 13:41:34.225534 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221653 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 13:41:34.225534 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221656 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 13:41:34.226087 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.221661 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 13:41:34.226087 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221824 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 13:41:34.226087 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221830 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 13:41:34.226087 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221833 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 13:41:34.226087 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221836 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 13:41:34.226087 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221839 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 13:41:34.226087 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221842 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 13:41:34.226087 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221845 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 13:41:34.226087 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221847 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 13:41:34.226087 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221850 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 13:41:34.226087 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221852 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 13:41:34.226087 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221855 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 13:41:34.226087 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221858 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 13:41:34.226087 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221861 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 13:41:34.226443 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221863 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 13:41:34.226443 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221866 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 13:41:34.226443 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221868 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 13:41:34.226443 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221871 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 13:41:34.226443 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221873 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 13:41:34.226443 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221876 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 13:41:34.226443 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221879 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 13:41:34.226443 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221881 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 13:41:34.226443 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221884 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 13:41:34.226443 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221886 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 13:41:34.226443 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221889 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 13:41:34.226443 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221892 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 13:41:34.226443 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221895 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 13:41:34.226443 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221898 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 13:41:34.226443 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221900 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 13:41:34.226443 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221923 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 13:41:34.226443 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221926 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 13:41:34.226443 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221929 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 13:41:34.226443 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221931 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 13:41:34.226443 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221935 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 13:41:34.226918 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221938 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 13:41:34.226918 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221940 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 13:41:34.226918 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221943 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 13:41:34.226918 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221945 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 13:41:34.226918 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221948 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 13:41:34.226918 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221950 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 13:41:34.226918 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221953 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 13:41:34.226918 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221956 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 13:41:34.226918 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221958 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 13:41:34.226918 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221961 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 13:41:34.226918 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221963 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 13:41:34.226918 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221966 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 13:41:34.226918 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221983 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 13:41:34.226918 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221986 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 13:41:34.226918 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221988 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 13:41:34.226918 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221991 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 13:41:34.226918 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221994 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 13:41:34.226918 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221996 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 13:41:34.226918 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.221999 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 13:41:34.226918 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222002 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 13:41:34.227428 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222004 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 13:41:34.227428 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222007 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 13:41:34.227428 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222009 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 13:41:34.227428 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222012 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 13:41:34.227428 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222015 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 13:41:34.227428 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222018 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 13:41:34.227428 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222021 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 13:41:34.227428 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222023 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 13:41:34.227428 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222040 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 13:41:34.227428 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222045 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 13:41:34.227428 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222048 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 13:41:34.227428 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222051 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 13:41:34.227428 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222054 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 13:41:34.227428 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222057 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 13:41:34.227428 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222060 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 13:41:34.227428 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222062 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 13:41:34.227428 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222065 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 13:41:34.227428 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222068 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 13:41:34.227428 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222070 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 13:41:34.227894 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222073 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 13:41:34.227894 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222075 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 13:41:34.227894 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222078 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 13:41:34.227894 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222081 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 13:41:34.227894 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222083 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 13:41:34.227894 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222086 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 13:41:34.227894 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222089 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 13:41:34.227894 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222091 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 13:41:34.227894 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222095 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 13:41:34.227894 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222098 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 13:41:34.227894 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222102 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 13:41:34.227894 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222105 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 13:41:34.227894 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222108 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 13:41:34.227894 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222112 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 13:41:34.227894 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222115 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 13:41:34.227894 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222118 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 13:41:34.227894 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222121 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 13:41:34.227894 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222124 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 13:41:34.227894 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222127 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 02 13:41:34.228377 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222129 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 13:41:34.228377 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222132 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 13:41:34.228377 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222135 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 13:41:34.228377 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222145 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 13:41:34.228377 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222148 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 13:41:34.228377 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222151 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 13:41:34.228377 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222153 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 13:41:34.228377 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:34.222156 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 13:41:34.228377 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.222162 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 13:41:34.228377 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.222316 2561 server.go:962] "Client rotation is on, will bootstrap in background" Jul 02 13:41:34.228377 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.225382 2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 02 13:41:34.229164 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.229145 2561 server.go:1019] "Starting client certificate rotation" Jul 02 13:41:34.229286 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.229266 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 13:41:34.229337 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.229324 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 13:41:34.232090 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.232070 2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 13:41:34.234501 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.234478 2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 13:41:34.247584 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.247565 2561 log.go:25] "Validated CRI v1 runtime API" Jul 02 13:41:34.249676 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.249662 2561 log.go:25] "Validated CRI v1 image API" Jul 02 13:41:34.250382 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.250368 2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 02 13:41:34.253051 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.253031 2561 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9f7aecb1-d95d-4d80-b633-e4d94fd768e6:/dev/nvme0n1p4 ea504fc0-a44e-4950-b8ef-284ef4840b4a:/dev/nvme0n1p3] Jul 02 13:41:34.253125 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.253049 2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 02 13:41:34.258744 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.258720 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 13:41:34.258882 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.258731 2561 manager.go:216] Machine: {Timestamp:2026-07-02 13:41:34.258241422 +0000 UTC m=+0.384377917 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100101 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27aa7d818c2ce2d0a1646b2aceb212 SystemUUID:ec27aa7d-818c-2ce2-d0a1-646b2aceb212 BootID:ac30d2c6-afb2-4c27-acaf-f4229482bffa Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8c:37:92:59:3d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8c:37:92:59:3d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:46:36:eb:25:8c:ee Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 02 13:41:34.258995 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.258883 2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 02 13:41:34.259059 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.259030 2561 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 02 13:41:34.262475 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.262443 2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 02 13:41:34.262661 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.262478 2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-112.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 02 13:41:34.263888 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.263874 2561 topology_manager.go:138] "Creating topology manager with none policy" Jul 02 13:41:34.263949 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.263894 2561 container_manager_linux.go:309] "Creating device plugin manager" Jul 02 13:41:34.263949 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.263906 2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 13:41:34.265007 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.264994 2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 13:41:34.266405 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.266394 2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 02 13:41:34.270878 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.270858 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 02 13:41:34.271058 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.271046 2561 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 02 13:41:34.271117 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.271096 2561 kubelet.go:486] "Attempting to sync node with API server" Jul 02 13:41:34.271173 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.271118 2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 02 13:41:34.271173 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.271140 2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 02 13:41:34.271173 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.271155 2561 kubelet.go:398] "Adding apiserver pod source" Jul 02 13:41:34.271173 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.271170 2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 02 13:41:34.271590 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.271575 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 13:41:34.271663 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.271599 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 13:41:34.274181 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.274160 2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 02 13:41:34.277077 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.277054 2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 02 13:41:34.277161 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.277102 2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 02 13:41:34.277934 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.277919 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 02 13:41:34.278014 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.277938 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 02 13:41:34.278014 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.277953 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 02 13:41:34.278014 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.277959 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 02 13:41:34.278014 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.277984 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 02 13:41:34.278014 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.277998 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 02 13:41:34.278014 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.278007 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 02 13:41:34.278014 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.278013 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 02 13:41:34.278198 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.278027 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 02 13:41:34.278198 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.278033 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 02 13:41:34.278198 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.278043 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 02 13:41:34.278198 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.278048 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 02 13:41:34.279270 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.279252 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 02 13:41:34.279339 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.279280 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 02 13:41:34.281222 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.281199 2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-112.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 02 13:41:34.281297 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:34.281235 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 02 13:41:34.281297 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:34.281235 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-112.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 02 13:41:34.281557 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.281540 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vsp9z" Jul 02 13:41:34.282545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.282532 2561 server.go:1268] "Started kubelet" Jul 02 13:41:34.282610 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.282585 2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 02 13:41:34.282717 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.282664 2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 02 13:41:34.282766 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.282739 2561 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 02 13:41:34.283749 ip-10-0-141-112 systemd[1]: Started Kubernetes Kubelet. Jul 02 13:41:34.283855 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.283804 2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 02 13:41:34.285707 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.285693 2561 server.go:310] "Adding debug handlers to kubelet server" Jul 02 13:41:34.288808 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.288789 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vsp9z" Jul 02 13:41:34.291120 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.291100 2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 02 13:41:34.291120 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.291112 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 02 13:41:34.291330 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:34.291261 2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 02 13:41:34.291389 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:34.291356 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-112.ec2.internal\" not found" Jul 02 13:41:34.291903 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.291887 2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 02 13:41:34.291903 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.291890 2561 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 02 13:41:34.292047 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.291916 2561 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 02 13:41:34.292096 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.292063 2561 reconstruct.go:97] "Volume reconstruction finished" Jul 02 13:41:34.292096 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.292072 2561 reconciler.go:29] "Reconciler: start to sync state" Jul 02 13:41:34.292888 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.292872 2561 factory.go:55] Registering systemd factory Jul 02 13:41:34.292888 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.292889 2561 factory.go:223] Registration of the systemd container factory successfully Jul 02 13:41:34.293100 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.293087 2561 factory.go:153] Registering CRI-O factory Jul 02 13:41:34.293100 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.293098 2561 factory.go:223] Registration of the crio container factory successfully Jul 02 13:41:34.293237 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.293144 2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 02 13:41:34.293237 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.293165 2561 factory.go:103] Registering Raw factory Jul 02 13:41:34.293237 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.293181 2561 manager.go:1196] Started watching for new ooms in manager Jul 02 13:41:34.293517 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.293505 2561 manager.go:349] Starting recovery of all containers Jul 02 13:41:34.297583 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.297563 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 13:41:34.304514 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.304118 2561 manager.go:354] Recovery completed Jul 02 13:41:34.305084 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:34.305062 2561 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-112.ec2.internal\" not found" node="ip-10-0-141-112.ec2.internal" Jul 02 13:41:34.310426 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.310412 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 13:41:34.312414 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.312398 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeHasSufficientMemory" Jul 02 13:41:34.312492 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.312433 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 13:41:34.312492 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.312470 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeHasSufficientPID" Jul 02 13:41:34.313038 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.313023 2561 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 02 13:41:34.313038 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.313035 2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 02 13:41:34.313167 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.313050 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 02 13:41:34.313167 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.313050 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 13:41:34.313799 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.313787 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeHasSufficientMemory" Jul 02 13:41:34.313858 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.313809 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 13:41:34.313858 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.313819 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeHasSufficientPID" Jul 02 13:41:34.315021 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.315009 2561 policy_none.go:49] "None policy: Start" Jul 02 13:41:34.315065 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.315026 2561 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 02 13:41:34.315065 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.315037 2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 02 13:41:34.316936 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.316925 2561 policy_none.go:47] "Start" Jul 02 13:41:34.355794 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.355612 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 02 13:41:34.371718 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.356992 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 02 13:41:34.371718 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.357030 2561 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 02 13:41:34.371718 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.357055 2561 kubelet.go:2448] "Starting kubelet main sync loop" Jul 02 13:41:34.371718 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:34.357094 2561 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 02 13:41:34.371718 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.359709 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 13:41:34.371718 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.360199 2561 manager.go:337] "Starting Device Plugin manager" Jul 02 13:41:34.371718 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:34.360234 2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 02 13:41:34.371718 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.360247 2561 server.go:87] "Starting device plugin registration server" Jul 02 13:41:34.371718 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.360538 2561 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 02 13:41:34.371718 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.360551 2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 02 13:41:34.371718 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.360664 2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 02 13:41:34.371718 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.360745 2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 02 13:41:34.371718 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.360754 2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 02 13:41:34.371718 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:34.361273 2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 02 13:41:34.371718 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:34.361323 2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-112.ec2.internal\" not found" Jul 02 13:41:34.457788 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.457689 2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-112.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-112.ec2.internal"] Jul 02 13:41:34.457788 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.457794 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 13:41:34.458775 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.458758 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeHasSufficientMemory" Jul 02 13:41:34.458873 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.458787 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 13:41:34.458873 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.458798 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeHasSufficientPID" Jul 02 13:41:34.460883 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.460868 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 13:41:34.461210 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.461196 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 13:41:34.461291 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.461277 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-112.ec2.internal" Jul 02 13:41:34.461328 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.461306 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 13:41:34.462064 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.462048 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeHasSufficientMemory" Jul 02 13:41:34.462152 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.462084 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 13:41:34.462152 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.462103 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeHasSufficientPID" Jul 02 13:41:34.462152 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.462048 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeHasSufficientMemory" Jul 02 13:41:34.462284 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.462156 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 13:41:34.462284 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.462168 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeHasSufficientPID" Jul 02 13:41:34.462677 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.462660 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeHasSufficientMemory" Jul 02 13:41:34.462785 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.462686 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 13:41:34.462785 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.462705 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeHasSufficientPID" Jul 02 13:41:34.462785 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.462728 2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-112.ec2.internal" Jul 02 13:41:34.464366 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.464349 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-112.ec2.internal" Jul 02 13:41:34.464446 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.464375 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 13:41:34.465109 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.465090 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeHasSufficientMemory" Jul 02 13:41:34.465197 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.465122 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 13:41:34.465197 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.465136 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeHasSufficientPID" Jul 02 13:41:34.471665 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.471649 2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-112.ec2.internal" Jul 02 13:41:34.471743 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:34.471673 2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-112.ec2.internal\": node \"ip-10-0-141-112.ec2.internal\" not found" Jul 02 13:41:34.480674 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:34.480653 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-112.ec2.internal\" not found" node="ip-10-0-141-112.ec2.internal" Jul 02 13:41:34.484305 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:34.484288 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-112.ec2.internal\" not found" node="ip-10-0-141-112.ec2.internal" Jul 02 13:41:34.485511 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:34.485496 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-112.ec2.internal\" not found" Jul 02 13:41:34.585601 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:34.585572 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-112.ec2.internal\" not found" Jul 02 13:41:34.592881 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.592857 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f61c5e3d2a1741508891516fc764b65e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-112.ec2.internal\" (UID: \"f61c5e3d2a1741508891516fc764b65e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-112.ec2.internal" Jul 02 13:41:34.592956 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.592884 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f61c5e3d2a1741508891516fc764b65e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-112.ec2.internal\" (UID: \"f61c5e3d2a1741508891516fc764b65e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-112.ec2.internal" Jul 02 13:41:34.592956 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.592902 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8f7c1481fcbae9bd1ac61a7f8b2618a4-config\") pod \"kube-apiserver-proxy-ip-10-0-141-112.ec2.internal\" (UID: \"8f7c1481fcbae9bd1ac61a7f8b2618a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-112.ec2.internal" Jul 02 13:41:34.686323 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:34.686285 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-112.ec2.internal\" not found" Jul 02 13:41:34.693555 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.693537 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8f7c1481fcbae9bd1ac61a7f8b2618a4-config\") pod \"kube-apiserver-proxy-ip-10-0-141-112.ec2.internal\" (UID: \"8f7c1481fcbae9bd1ac61a7f8b2618a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-112.ec2.internal" Jul 02 13:41:34.693618 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.693572 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f61c5e3d2a1741508891516fc764b65e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-112.ec2.internal\" (UID: \"f61c5e3d2a1741508891516fc764b65e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-112.ec2.internal" Jul 02 13:41:34.693618 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.693589 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f61c5e3d2a1741508891516fc764b65e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-112.ec2.internal\" (UID: \"f61c5e3d2a1741508891516fc764b65e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-112.ec2.internal" Jul 02 13:41:34.693618 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.693615 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f61c5e3d2a1741508891516fc764b65e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-112.ec2.internal\" (UID: \"f61c5e3d2a1741508891516fc764b65e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-112.ec2.internal" Jul 02 13:41:34.693718 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.693540 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8f7c1481fcbae9bd1ac61a7f8b2618a4-config\") pod \"kube-apiserver-proxy-ip-10-0-141-112.ec2.internal\" (UID: \"8f7c1481fcbae9bd1ac61a7f8b2618a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-112.ec2.internal" Jul 02 13:41:34.693718 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.693659 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f61c5e3d2a1741508891516fc764b65e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-112.ec2.internal\" (UID: \"f61c5e3d2a1741508891516fc764b65e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-112.ec2.internal" Jul 02 13:41:34.786081 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.786001 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-112.ec2.internal" Jul 02 13:41:34.787061 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:34.787043 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-112.ec2.internal\" not found" Jul 02 13:41:34.788167 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:34.788141 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-112.ec2.internal" Jul 02 13:41:34.887758 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:34.887719 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-112.ec2.internal\" not found" Jul 02 13:41:34.988325 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:34.988289 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-112.ec2.internal\" not found" Jul 02 13:41:35.088852 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:35.088780 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-112.ec2.internal\" not found" Jul 02 13:41:35.189309 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:35.189278 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-112.ec2.internal\" not found" Jul 02 13:41:35.226744 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:35.226707 2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 02 13:41:35.227490 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:35.226893 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 13:41:35.227490 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:35.226916 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 13:41:35.290171 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:35.290146 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-112.ec2.internal\" not found" Jul 02 13:41:35.295493 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:35.295460 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 13:36:34 +0000 UTC" deadline="2027-12-26 19:39:05.337669965 +0000 UTC" Jul 02 13:41:35.295493 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:35.295489 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13013h57m30.042184578s" Jul 02 13:41:35.331362 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:35.331332 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf61c5e3d2a1741508891516fc764b65e.slice/crio-4056580be84ce2b921d763dd052a354c91943de5815011c81456757b3ce0283e WatchSource:0}: Error finding container 4056580be84ce2b921d763dd052a354c91943de5815011c81456757b3ce0283e: Status 404 returned error can't find the container with id 4056580be84ce2b921d763dd052a354c91943de5815011c81456757b3ce0283e Jul 02 13:41:35.331697 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:35.331675 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f7c1481fcbae9bd1ac61a7f8b2618a4.slice/crio-6f9891c9bec356cefa4f1008eb2d8301cb13d53a9c873ecc5c390aedb71a90d1 WatchSource:0}: Error finding container 6f9891c9bec356cefa4f1008eb2d8301cb13d53a9c873ecc5c390aedb71a90d1: Status 404 returned error can't find the container with id 6f9891c9bec356cefa4f1008eb2d8301cb13d53a9c873ecc5c390aedb71a90d1 Jul 02 13:41:35.337293 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:35.337276 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 13:41:35.361487 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:35.361441 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-112.ec2.internal" event={"ID":"8f7c1481fcbae9bd1ac61a7f8b2618a4","Type":"ContainerStarted","Data":"6f9891c9bec356cefa4f1008eb2d8301cb13d53a9c873ecc5c390aedb71a90d1"} Jul 02 13:41:35.362514 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:35.362490 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-112.ec2.internal" event={"ID":"f61c5e3d2a1741508891516fc764b65e","Type":"ContainerStarted","Data":"4056580be84ce2b921d763dd052a354c91943de5815011c81456757b3ce0283e"} Jul 02 13:41:35.390729 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:35.390702 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-112.ec2.internal\" not found" Jul 02 13:41:35.491261 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:35.491229 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-112.ec2.internal\" not found" Jul 02 13:41:35.591755 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:35.591690 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-112.ec2.internal\" not found" Jul 02 13:41:35.624368 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:35.624349 2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 13:41:35.692059 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:35.692025 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-112.ec2.internal" Jul 02 13:41:35.703368 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:35.703346 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 13:41:35.705516 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:35.705499 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-112.ec2.internal" Jul 02 13:41:35.715389 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:35.715354 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 13:41:35.739341 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:35.739317 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 13:41:36.272881 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.272849 2561 apiserver.go:52] "Watching apiserver" Jul 02 13:41:36.282592 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.282349 2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 02 13:41:36.283574 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.283527 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-x5pxb","openshift-dns/node-resolver-kcdlm","openshift-ovn-kubernetes/ovnkube-node-ppzmg","kube-system/global-pull-secret-syncer-flv4b","openshift-image-registry/node-ca-cpjcf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-112.ec2.internal","openshift-multus/multus-additional-cni-plugins-p66lc","openshift-multus/multus-xdsnn","openshift-multus/network-metrics-daemon-j9rh2","openshift-network-diagnostics/network-check-target-jv42p","openshift-network-operator/iptables-alerter-hz9sp","kube-system/konnectivity-agent-gp6w6","kube-system/kube-apiserver-proxy-ip-10-0-141-112.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4"] Jul 02 13:41:36.286295 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.286269 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.288860 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.288553 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hz9sp" Jul 02 13:41:36.288860 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.288591 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 02 13:41:36.289091 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.289067 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 02 13:41:36.289201 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.289077 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 02 13:41:36.289201 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.289142 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-r7fgh\"" Jul 02 13:41:36.289201 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.289173 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 02 13:41:36.289489 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.289471 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 02 13:41:36.291002 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.290982 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.291403 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.291301 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 02 13:41:36.291829 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.291798 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 02 13:41:36.291928 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.291912 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 02 13:41:36.292000 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.291966 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 02 13:41:36.292182 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.292166 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-864hb\"" Jul 02 13:41:36.293476 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.293224 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tskfc\"" Jul 02 13:41:36.293476 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.293257 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 02 13:41:36.293476 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.293337 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 02 13:41:36.293476 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.293258 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 02 13:41:36.293722 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.293521 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 02 13:41:36.293722 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.293593 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 02 13:41:36.294114 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.293861 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:36.294114 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:36.293935 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:41:36.294114 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.294007 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 02 13:41:36.296074 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.296050 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cpjcf" Jul 02 13:41:36.298728 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.298350 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 02 13:41:36.298728 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.298469 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 02 13:41:36.298728 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.298571 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 02 13:41:36.298728 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.298715 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ffn6m\"" Jul 02 13:41:36.298996 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.298824 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.300934 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.300912 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 02 13:41:36.301085 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.301064 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 02 13:41:36.301154 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.301116 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 02 13:41:36.301231 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.301215 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-v9v6c\"" Jul 02 13:41:36.301460 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.301440 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 13:41:36.301728 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.301707 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-system-cni-dir\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.301818 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.301769 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:36.301873 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.301818 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-run-systemd\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.301923 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:36.301856 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:41:36.301923 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.301852 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-var-lib-openvswitch\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.302049 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.301928 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-ovnkube-script-lib\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.302049 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.301963 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqsd6\" (UniqueName: \"kubernetes.io/projected/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-kube-api-access-mqsd6\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.302049 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302014 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a5b42e80-1152-44f6-8d39-fca751ae0e46-iptables-alerter-script\") pod \"iptables-alerter-hz9sp\" (UID: \"a5b42e80-1152-44f6-8d39-fca751ae0e46\") " pod="openshift-network-operator/iptables-alerter-hz9sp" Jul 02 13:41:36.302049 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302038 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-node-log\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.302236 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302061 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-host-run-ovn-kubernetes\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.302236 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302099 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.302236 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302138 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kfpg\" (UniqueName: \"kubernetes.io/projected/a5b42e80-1152-44f6-8d39-fca751ae0e46-kube-api-access-6kfpg\") pod \"iptables-alerter-hz9sp\" (UID: \"a5b42e80-1152-44f6-8d39-fca751ae0e46\") " pod="openshift-network-operator/iptables-alerter-hz9sp" Jul 02 13:41:36.302236 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302164 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-run-openvswitch\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.302236 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302189 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-log-socket\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.302236 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302218 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-env-overrides\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.302511 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302268 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-cnibin\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.302511 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302293 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-os-release\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.302511 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302335 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a5b42e80-1152-44f6-8d39-fca751ae0e46-host-slash\") pod \"iptables-alerter-hz9sp\" (UID: \"a5b42e80-1152-44f6-8d39-fca751ae0e46\") " pod="openshift-network-operator/iptables-alerter-hz9sp" Jul 02 13:41:36.302511 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302369 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-host-kubelet\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.302511 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302394 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-etc-openvswitch\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.302511 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302419 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-kubelet-config\") pod \"global-pull-secret-syncer-flv4b\" (UID: \"cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f\") " pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:36.302511 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302448 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-cni-binary-copy\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.302511 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302502 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-host-slash\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.302901 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302525 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-systemd-units\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.302901 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302545 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-host-cni-netd\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.302901 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302572 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.302901 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302609 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-ovnkube-config\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.302901 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302637 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret\") pod \"global-pull-secret-syncer-flv4b\" (UID: \"cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f\") " pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:36.302901 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302661 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-host-cni-bin\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.302901 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302685 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.302901 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302735 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.302901 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302773 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-run-ovn\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.302901 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302827 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-ovn-node-metrics-cert\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.302901 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302867 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmmqn\" (UniqueName: \"kubernetes.io/projected/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-kube-api-access-mmmqn\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.302901 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302894 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-dbus\") pod \"global-pull-secret-syncer-flv4b\" (UID: \"cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f\") " pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:36.303431 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.302915 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-host-run-netns\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.305456 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.305436 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.307746 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.307576 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 02 13:41:36.307746 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.307630 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pq58h\"" Jul 02 13:41:36.308098 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.308081 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kcdlm" Jul 02 13:41:36.310476 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.310455 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 02 13:41:36.310862 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.310839 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 02 13:41:36.311369 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.311211 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-x7dmx\"" Jul 02 13:41:36.314678 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.314135 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:41:36.314678 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:36.314349 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:41:36.317091 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.317067 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.319230 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.319206 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 02 13:41:36.319330 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.319261 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fpr7r\"" Jul 02 13:41:36.319402 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.319208 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 02 13:41:36.319465 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.319450 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gp6w6" Jul 02 13:41:36.320943 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.320907 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fkvjg" Jul 02 13:41:36.322075 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.322049 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 02 13:41:36.322257 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.322155 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lwz8h\"" Jul 02 13:41:36.322424 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.322201 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 02 13:41:36.327520 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.327499 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fkvjg" Jul 02 13:41:36.393579 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.393547 2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 02 13:41:36.403202 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403154 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-log-socket\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.403202 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403206 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-env-overrides\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.403443 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403238 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5a0abd99-eab7-4570-bfee-9c54ac664871-sys-fs\") pod \"aws-ebs-csi-driver-node-7dwf4\" (UID: \"5a0abd99-eab7-4570-bfee-9c54ac664871\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.403443 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403263 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-os-release\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.403443 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403286 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-cni-binary-copy\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.403443 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403307 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-hostroot\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.403443 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403331 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-multus-daemon-config\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.403443 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403359 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-os-release\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.403443 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403386 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs\") pod \"network-metrics-daemon-j9rh2\" (UID: \"da0298d6-5c20-459a-864b-0a3146e256ee\") " pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:36.403443 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403411 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-host-var-lib-cni-multus\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.403810 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403446 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-etc-kubernetes\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.403810 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403470 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-498vp\" (UniqueName: \"kubernetes.io/projected/c28ced17-5ab2-4167-944a-91104427887e-kube-api-access-498vp\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.403810 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403492 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1aec8ab3-45e7-422c-985b-0f5541ed6149-host\") pod \"node-ca-cpjcf\" (UID: \"1aec8ab3-45e7-422c-985b-0f5541ed6149\") " pod="openshift-image-registry/node-ca-cpjcf" Jul 02 13:41:36.403810 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403515 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5a0abd99-eab7-4570-bfee-9c54ac664871-socket-dir\") pod \"aws-ebs-csi-driver-node-7dwf4\" (UID: \"5a0abd99-eab7-4570-bfee-9c54ac664871\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.403810 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403539 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-etc-sysconfig\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.403810 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403561 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-etc-sysctl-d\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.403810 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403585 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-lib-modules\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.403810 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403613 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-host-cni-netd\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.403810 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403637 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.403810 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403665 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-host-run-netns\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.403810 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403689 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-host-var-lib-kubelet\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.403810 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403727 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-etc-sysctl-conf\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.403810 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403753 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d160c6f-4496-4913-a38a-26f61cd3effe-tmp-dir\") pod \"node-resolver-kcdlm\" (UID: \"4d160c6f-4496-4913-a38a-26f61cd3effe\") " pod="openshift-dns/node-resolver-kcdlm" Jul 02 13:41:36.403810 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403775 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-multus-cni-dir\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.403810 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403811 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-host-cni-bin\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.404491 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403854 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.404491 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403881 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-run-ovn\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.404491 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403909 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmmqn\" (UniqueName: \"kubernetes.io/projected/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-kube-api-access-mmmqn\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.404491 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403934 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5a0abd99-eab7-4570-bfee-9c54ac664871-registration-dir\") pod \"aws-ebs-csi-driver-node-7dwf4\" (UID: \"5a0abd99-eab7-4570-bfee-9c54ac664871\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.404491 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403959 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-system-cni-dir\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.404491 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.403999 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-run-systemd\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.404491 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404024 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-var-lib-openvswitch\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.404491 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404061 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq8hh\" (UniqueName: \"kubernetes.io/projected/da0298d6-5c20-459a-864b-0a3146e256ee-kube-api-access-hq8hh\") pod \"network-metrics-daemon-j9rh2\" (UID: \"da0298d6-5c20-459a-864b-0a3146e256ee\") " pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:36.404491 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404086 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-host-var-lib-cni-bin\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.404491 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404155 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-multus-conf-dir\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.404491 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404179 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-etc-kubernetes\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.404491 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404202 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bf8c5897-0051-41d8-bc25-2f990a7dd74b-agent-certs\") pod \"konnectivity-agent-gp6w6\" (UID: \"bf8c5897-0051-41d8-bc25-2f990a7dd74b\") " pod="kube-system/konnectivity-agent-gp6w6" Jul 02 13:41:36.404491 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404238 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqsd6\" (UniqueName: \"kubernetes.io/projected/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-kube-api-access-mqsd6\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.404491 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404264 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a5b42e80-1152-44f6-8d39-fca751ae0e46-iptables-alerter-script\") pod \"iptables-alerter-hz9sp\" (UID: \"a5b42e80-1152-44f6-8d39-fca751ae0e46\") " pod="openshift-network-operator/iptables-alerter-hz9sp" Jul 02 13:41:36.404491 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404298 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-node-log\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.404491 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404333 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1aec8ab3-45e7-422c-985b-0f5541ed6149-serviceca\") pod \"node-ca-cpjcf\" (UID: \"1aec8ab3-45e7-422c-985b-0f5541ed6149\") " pod="openshift-image-registry/node-ca-cpjcf" Jul 02 13:41:36.405217 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404341 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.405217 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404361 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxnv4\" (UniqueName: \"kubernetes.io/projected/1aec8ab3-45e7-422c-985b-0f5541ed6149-kube-api-access-wxnv4\") pod \"node-ca-cpjcf\" (UID: \"1aec8ab3-45e7-422c-985b-0f5541ed6149\") " pod="openshift-image-registry/node-ca-cpjcf" Jul 02 13:41:36.405217 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404375 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-log-socket\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.405217 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404415 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-cnibin\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.405217 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404441 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-host-run-k8s-cni-cncf-io\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.405217 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404538 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gsz6\" (UniqueName: \"kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6\") pod \"network-check-target-jv42p\" (UID: \"684ebf83-fb83-47a1-94c7-ecd7bba444c7\") " pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:41:36.405217 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404568 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-host-run-ovn-kubernetes\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.405217 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404613 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.405217 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404640 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kfpg\" (UniqueName: \"kubernetes.io/projected/a5b42e80-1152-44f6-8d39-fca751ae0e46-kube-api-access-6kfpg\") pod \"iptables-alerter-hz9sp\" (UID: \"a5b42e80-1152-44f6-8d39-fca751ae0e46\") " pod="openshift-network-operator/iptables-alerter-hz9sp" Jul 02 13:41:36.405217 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404677 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-etc-modprobe-d\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.405217 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404700 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c28ced17-5ab2-4167-944a-91104427887e-etc-tuned\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.405217 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404723 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bf8c5897-0051-41d8-bc25-2f990a7dd74b-konnectivity-ca\") pod \"konnectivity-agent-gp6w6\" (UID: \"bf8c5897-0051-41d8-bc25-2f990a7dd74b\") " pod="kube-system/konnectivity-agent-gp6w6" Jul 02 13:41:36.405217 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404778 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-cnibin\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.405217 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404812 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a5b42e80-1152-44f6-8d39-fca751ae0e46-host-slash\") pod \"iptables-alerter-hz9sp\" (UID: \"a5b42e80-1152-44f6-8d39-fca751ae0e46\") " pod="openshift-network-operator/iptables-alerter-hz9sp" Jul 02 13:41:36.405217 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404839 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-host-kubelet\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.405217 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404875 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-etc-openvswitch\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.405217 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.404930 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-kubelet-config\") pod \"global-pull-secret-syncer-flv4b\" (UID: \"cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f\") " pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:36.405993 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405030 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-env-overrides\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.405993 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405187 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-node-log\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.405993 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405200 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5a0abd99-eab7-4570-bfee-9c54ac664871-device-dir\") pod \"aws-ebs-csi-driver-node-7dwf4\" (UID: \"5a0abd99-eab7-4570-bfee-9c54ac664871\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.405993 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405221 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-etc-openvswitch\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.405993 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405254 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnn4c\" (UniqueName: \"kubernetes.io/projected/5a0abd99-eab7-4570-bfee-9c54ac664871-kube-api-access-hnn4c\") pod \"aws-ebs-csi-driver-node-7dwf4\" (UID: \"5a0abd99-eab7-4570-bfee-9c54ac664871\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.405993 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405274 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-var-lib-openvswitch\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.405993 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405259 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-system-cni-dir\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.405993 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405297 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.405993 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405310 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4d160c6f-4496-4913-a38a-26f61cd3effe-hosts-file\") pod \"node-resolver-kcdlm\" (UID: \"4d160c6f-4496-4913-a38a-26f61cd3effe\") " pod="openshift-dns/node-resolver-kcdlm" Jul 02 13:41:36.405993 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405320 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-kubelet-config\") pod \"global-pull-secret-syncer-flv4b\" (UID: \"cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f\") " pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:36.405993 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405341 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-cni-binary-copy\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.405993 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405375 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-host-cni-bin\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.405993 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405379 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-os-release\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.405993 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405393 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-host-slash\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.405993 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405422 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-host-cni-netd\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.405993 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405435 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wmzl\" (UniqueName: \"kubernetes.io/projected/4d160c6f-4496-4913-a38a-26f61cd3effe-kube-api-access-7wmzl\") pod \"node-resolver-kcdlm\" (UID: \"4d160c6f-4496-4913-a38a-26f61cd3effe\") " pod="openshift-dns/node-resolver-kcdlm" Jul 02 13:41:36.405993 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405503 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a5b42e80-1152-44f6-8d39-fca751ae0e46-host-slash\") pod \"iptables-alerter-hz9sp\" (UID: \"a5b42e80-1152-44f6-8d39-fca751ae0e46\") " pod="openshift-network-operator/iptables-alerter-hz9sp" Jul 02 13:41:36.406776 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405505 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-run-ovn\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.406776 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405546 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-multus-socket-dir-parent\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.406776 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405562 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-host-slash\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.406776 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405584 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-host-run-multus-certs\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.406776 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405600 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-cnibin\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.406776 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405611 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-etc-systemd\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.406776 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405636 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-sys\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.406776 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405647 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-host-run-ovn-kubernetes\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.406776 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405678 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c28ced17-5ab2-4167-944a-91104427887e-tmp\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.406776 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405686 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-host-kubelet\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.406776 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405720 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-systemd-units\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.406776 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405748 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-ovnkube-config\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.406776 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405780 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret\") pod \"global-pull-secret-syncer-flv4b\" (UID: \"cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f\") " pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:36.406776 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405810 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5a0abd99-eab7-4570-bfee-9c54ac664871-etc-selinux\") pod \"aws-ebs-csi-driver-node-7dwf4\" (UID: \"5a0abd99-eab7-4570-bfee-9c54ac664871\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.406776 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405835 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-host\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.406776 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405878 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5a0abd99-eab7-4570-bfee-9c54ac664871-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7dwf4\" (UID: \"5a0abd99-eab7-4570-bfee-9c54ac664871\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.406776 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405906 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.407455 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405925 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a5b42e80-1152-44f6-8d39-fca751ae0e46-iptables-alerter-script\") pod \"iptables-alerter-hz9sp\" (UID: \"a5b42e80-1152-44f6-8d39-fca751ae0e46\") " pod="openshift-network-operator/iptables-alerter-hz9sp" Jul 02 13:41:36.407455 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405930 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.407455 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405949 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-systemd-units\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.407455 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.405949 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-ovn-node-metrics-cert\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.407455 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:36.406078 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 13:41:36.407455 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:36.406168 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret podName:cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f nodeName:}" failed. No retries permitted until 2026-07-02 13:41:36.906128687 +0000 UTC m=+3.032265185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret") pod "global-pull-secret-syncer-flv4b" (UID: "cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f") : object "kube-system"/"original-pull-secret" not registered Jul 02 13:41:36.407455 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.406198 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-cni-binary-copy\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.407455 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.406260 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-dbus\") pod \"global-pull-secret-syncer-flv4b\" (UID: \"cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f\") " pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:36.407455 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.406298 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-run\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.407455 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.406335 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-host-run-netns\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.407455 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.406338 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-ovnkube-config\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.407455 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.406345 2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 02 13:41:36.407455 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.406364 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-system-cni-dir\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.407455 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.406404 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-host-run-netns\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.407455 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.406442 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-ovnkube-script-lib\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.407455 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.406493 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-dbus\") pod \"global-pull-secret-syncer-flv4b\" (UID: \"cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f\") " pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:36.407455 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.406511 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-var-lib-kubelet\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.408204 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.406553 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fckz5\" (UniqueName: \"kubernetes.io/projected/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-kube-api-access-fckz5\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.408204 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.406581 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.408204 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.406609 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-run-openvswitch\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.408204 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.406649 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-run-openvswitch\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.408564 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.408529 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-ovnkube-script-lib\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.411241 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.408889 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-run-systemd\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.411241 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.410526 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 13:41:36.411241 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.410938 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-ovn-node-metrics-cert\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.415748 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.415501 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqsd6\" (UniqueName: \"kubernetes.io/projected/6ac92ae9-c78b-4c00-ae36-41e1c2f5e692-kube-api-access-mqsd6\") pod \"multus-additional-cni-plugins-p66lc\" (UID: \"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692\") " pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.416227 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.415889 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmmqn\" (UniqueName: \"kubernetes.io/projected/ddf20f35-9cd8-406c-b529-8e2949a8d4e5-kube-api-access-mmmqn\") pod \"ovnkube-node-ppzmg\" (UID: \"ddf20f35-9cd8-406c-b529-8e2949a8d4e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.416352 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.416331 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kfpg\" (UniqueName: \"kubernetes.io/projected/a5b42e80-1152-44f6-8d39-fca751ae0e46-kube-api-access-6kfpg\") pod \"iptables-alerter-hz9sp\" (UID: \"a5b42e80-1152-44f6-8d39-fca751ae0e46\") " pod="openshift-network-operator/iptables-alerter-hz9sp" Jul 02 13:41:36.507686 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.507648 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gsz6\" (UniqueName: \"kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6\") pod \"network-check-target-jv42p\" (UID: \"684ebf83-fb83-47a1-94c7-ecd7bba444c7\") " pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:41:36.507859 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.507696 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-etc-modprobe-d\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.507859 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.507721 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c28ced17-5ab2-4167-944a-91104427887e-etc-tuned\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.507859 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.507741 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bf8c5897-0051-41d8-bc25-2f990a7dd74b-konnectivity-ca\") pod \"konnectivity-agent-gp6w6\" (UID: \"bf8c5897-0051-41d8-bc25-2f990a7dd74b\") " pod="kube-system/konnectivity-agent-gp6w6" Jul 02 13:41:36.507859 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.507765 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5a0abd99-eab7-4570-bfee-9c54ac664871-device-dir\") pod \"aws-ebs-csi-driver-node-7dwf4\" (UID: \"5a0abd99-eab7-4570-bfee-9c54ac664871\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.507859 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.507787 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnn4c\" (UniqueName: \"kubernetes.io/projected/5a0abd99-eab7-4570-bfee-9c54ac664871-kube-api-access-hnn4c\") pod \"aws-ebs-csi-driver-node-7dwf4\" (UID: \"5a0abd99-eab7-4570-bfee-9c54ac664871\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.507859 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.507808 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4d160c6f-4496-4913-a38a-26f61cd3effe-hosts-file\") pod \"node-resolver-kcdlm\" (UID: \"4d160c6f-4496-4913-a38a-26f61cd3effe\") " pod="openshift-dns/node-resolver-kcdlm" Jul 02 13:41:36.507859 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.507859 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wmzl\" (UniqueName: \"kubernetes.io/projected/4d160c6f-4496-4913-a38a-26f61cd3effe-kube-api-access-7wmzl\") pod \"node-resolver-kcdlm\" (UID: \"4d160c6f-4496-4913-a38a-26f61cd3effe\") " pod="openshift-dns/node-resolver-kcdlm" Jul 02 13:41:36.508266 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.507885 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-multus-socket-dir-parent\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.508266 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.507921 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-host-run-multus-certs\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.508266 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.507943 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-etc-systemd\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.508266 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.507967 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-sys\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.508266 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508006 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c28ced17-5ab2-4167-944a-91104427887e-tmp\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.508266 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508033 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4d160c6f-4496-4913-a38a-26f61cd3effe-hosts-file\") pod \"node-resolver-kcdlm\" (UID: \"4d160c6f-4496-4913-a38a-26f61cd3effe\") " pod="openshift-dns/node-resolver-kcdlm" Jul 02 13:41:36.508266 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508045 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5a0abd99-eab7-4570-bfee-9c54ac664871-etc-selinux\") pod \"aws-ebs-csi-driver-node-7dwf4\" (UID: \"5a0abd99-eab7-4570-bfee-9c54ac664871\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.508266 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508125 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5a0abd99-eab7-4570-bfee-9c54ac664871-device-dir\") pod \"aws-ebs-csi-driver-node-7dwf4\" (UID: \"5a0abd99-eab7-4570-bfee-9c54ac664871\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.508266 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508173 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-host\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.508266 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508203 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5a0abd99-eab7-4570-bfee-9c54ac664871-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7dwf4\" (UID: \"5a0abd99-eab7-4570-bfee-9c54ac664871\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.508266 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508265 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-multus-socket-dir-parent\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.508762 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508233 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-run\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.508762 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508317 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-host-run-multus-certs\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.508762 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508332 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-run\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.508762 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508346 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-system-cni-dir\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.508762 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508373 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-var-lib-kubelet\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.508762 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508398 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5a0abd99-eab7-4570-bfee-9c54ac664871-etc-selinux\") pod \"aws-ebs-csi-driver-node-7dwf4\" (UID: \"5a0abd99-eab7-4570-bfee-9c54ac664871\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.508762 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508415 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fckz5\" (UniqueName: \"kubernetes.io/projected/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-kube-api-access-fckz5\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.508762 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508514 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-etc-modprobe-d\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.508762 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508546 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5a0abd99-eab7-4570-bfee-9c54ac664871-sys-fs\") pod \"aws-ebs-csi-driver-node-7dwf4\" (UID: \"5a0abd99-eab7-4570-bfee-9c54ac664871\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.508762 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508570 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-os-release\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.508762 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508593 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-cni-binary-copy\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.508762 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508636 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-hostroot\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.508762 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508660 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-multus-daemon-config\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.508762 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508693 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs\") pod \"network-metrics-daemon-j9rh2\" (UID: \"da0298d6-5c20-459a-864b-0a3146e256ee\") " pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:36.508762 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508696 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-os-release\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.508762 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508719 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-host-var-lib-cni-multus\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.508762 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508744 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-etc-kubernetes\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.508762 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508765 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-var-lib-kubelet\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.509598 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508767 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-498vp\" (UniqueName: \"kubernetes.io/projected/c28ced17-5ab2-4167-944a-91104427887e-kube-api-access-498vp\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.509598 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508814 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1aec8ab3-45e7-422c-985b-0f5541ed6149-host\") pod \"node-ca-cpjcf\" (UID: \"1aec8ab3-45e7-422c-985b-0f5541ed6149\") " pod="openshift-image-registry/node-ca-cpjcf" Jul 02 13:41:36.509598 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508840 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5a0abd99-eab7-4570-bfee-9c54ac664871-socket-dir\") pod \"aws-ebs-csi-driver-node-7dwf4\" (UID: \"5a0abd99-eab7-4570-bfee-9c54ac664871\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.509598 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508849 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/bf8c5897-0051-41d8-bc25-2f990a7dd74b-konnectivity-ca\") pod \"konnectivity-agent-gp6w6\" (UID: \"bf8c5897-0051-41d8-bc25-2f990a7dd74b\") " pod="kube-system/konnectivity-agent-gp6w6" Jul 02 13:41:36.509598 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508866 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-etc-sysconfig\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.509598 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508895 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-etc-sysctl-d\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.509598 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508921 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5a0abd99-eab7-4570-bfee-9c54ac664871-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7dwf4\" (UID: \"5a0abd99-eab7-4570-bfee-9c54ac664871\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.509598 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508990 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1aec8ab3-45e7-422c-985b-0f5541ed6149-host\") pod \"node-ca-cpjcf\" (UID: \"1aec8ab3-45e7-422c-985b-0f5541ed6149\") " pod="openshift-image-registry/node-ca-cpjcf" Jul 02 13:41:36.509598 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.508990 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-sys\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.509598 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509003 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-system-cni-dir\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.509598 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509015 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5a0abd99-eab7-4570-bfee-9c54ac664871-socket-dir\") pod \"aws-ebs-csi-driver-node-7dwf4\" (UID: \"5a0abd99-eab7-4570-bfee-9c54ac664871\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.509598 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509048 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5a0abd99-eab7-4570-bfee-9c54ac664871-sys-fs\") pod \"aws-ebs-csi-driver-node-7dwf4\" (UID: \"5a0abd99-eab7-4570-bfee-9c54ac664871\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.509598 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509057 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-lib-modules\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.509598 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:36.509050 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:41:36.509598 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509108 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-etc-systemd\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.509598 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509127 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-etc-sysctl-d\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.509598 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:36.509135 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs podName:da0298d6-5c20-459a-864b-0a3146e256ee nodeName:}" failed. No retries permitted until 2026-07-02 13:41:37.009118607 +0000 UTC m=+3.135255094 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs") pod "network-metrics-daemon-j9rh2" (UID: "da0298d6-5c20-459a-864b-0a3146e256ee") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:41:36.510532 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509143 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-etc-kubernetes\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.510532 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509169 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-host-var-lib-cni-multus\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.510532 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509177 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-host-run-netns\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.510532 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509217 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-hostroot\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.510532 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509217 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-host-run-netns\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.510532 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509248 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-host-var-lib-kubelet\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.510532 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509290 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-etc-sysctl-conf\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.510532 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509303 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-lib-modules\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.510532 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509316 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d160c6f-4496-4913-a38a-26f61cd3effe-tmp-dir\") pod \"node-resolver-kcdlm\" (UID: \"4d160c6f-4496-4913-a38a-26f61cd3effe\") " pod="openshift-dns/node-resolver-kcdlm" Jul 02 13:41:36.510532 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509336 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-host-var-lib-kubelet\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.510532 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509341 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-multus-cni-dir\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.510532 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509393 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-multus-cni-dir\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.510532 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509398 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-etc-sysctl-conf\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.510532 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509456 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5a0abd99-eab7-4570-bfee-9c54ac664871-registration-dir\") pod \"aws-ebs-csi-driver-node-7dwf4\" (UID: \"5a0abd99-eab7-4570-bfee-9c54ac664871\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.510532 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509457 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-etc-sysconfig\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.510532 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509484 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-host\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.510532 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509489 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq8hh\" (UniqueName: \"kubernetes.io/projected/da0298d6-5c20-459a-864b-0a3146e256ee-kube-api-access-hq8hh\") pod \"network-metrics-daemon-j9rh2\" (UID: \"da0298d6-5c20-459a-864b-0a3146e256ee\") " pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:36.510532 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509514 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-host-var-lib-cni-bin\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.511400 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509536 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-multus-conf-dir\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.511400 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509557 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-etc-kubernetes\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.511400 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509579 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bf8c5897-0051-41d8-bc25-2f990a7dd74b-agent-certs\") pod \"konnectivity-agent-gp6w6\" (UID: \"bf8c5897-0051-41d8-bc25-2f990a7dd74b\") " pod="kube-system/konnectivity-agent-gp6w6" Jul 02 13:41:36.511400 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509603 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-multus-daemon-config\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.511400 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509608 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-cni-binary-copy\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.511400 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509650 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-host-var-lib-cni-bin\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.511400 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509671 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5a0abd99-eab7-4570-bfee-9c54ac664871-registration-dir\") pod \"aws-ebs-csi-driver-node-7dwf4\" (UID: \"5a0abd99-eab7-4570-bfee-9c54ac664871\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.511400 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509689 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-multus-conf-dir\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.511400 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509723 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c28ced17-5ab2-4167-944a-91104427887e-etc-kubernetes\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.511400 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509606 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1aec8ab3-45e7-422c-985b-0f5541ed6149-serviceca\") pod \"node-ca-cpjcf\" (UID: \"1aec8ab3-45e7-422c-985b-0f5541ed6149\") " pod="openshift-image-registry/node-ca-cpjcf" Jul 02 13:41:36.511400 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509742 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4d160c6f-4496-4913-a38a-26f61cd3effe-tmp-dir\") pod \"node-resolver-kcdlm\" (UID: \"4d160c6f-4496-4913-a38a-26f61cd3effe\") " pod="openshift-dns/node-resolver-kcdlm" Jul 02 13:41:36.511400 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509764 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxnv4\" (UniqueName: \"kubernetes.io/projected/1aec8ab3-45e7-422c-985b-0f5541ed6149-kube-api-access-wxnv4\") pod \"node-ca-cpjcf\" (UID: \"1aec8ab3-45e7-422c-985b-0f5541ed6149\") " pod="openshift-image-registry/node-ca-cpjcf" Jul 02 13:41:36.511400 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509791 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-cnibin\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.511400 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509815 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-host-run-k8s-cni-cncf-io\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.511400 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509875 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-host-run-k8s-cni-cncf-io\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.511400 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.509934 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-cnibin\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.511400 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.510019 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1aec8ab3-45e7-422c-985b-0f5541ed6149-serviceca\") pod \"node-ca-cpjcf\" (UID: \"1aec8ab3-45e7-422c-985b-0f5541ed6149\") " pod="openshift-image-registry/node-ca-cpjcf" Jul 02 13:41:36.512273 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.511883 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c28ced17-5ab2-4167-944a-91104427887e-etc-tuned\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.512273 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.512081 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c28ced17-5ab2-4167-944a-91104427887e-tmp\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.512527 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.512506 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/bf8c5897-0051-41d8-bc25-2f990a7dd74b-agent-certs\") pod \"konnectivity-agent-gp6w6\" (UID: \"bf8c5897-0051-41d8-bc25-2f990a7dd74b\") " pod="kube-system/konnectivity-agent-gp6w6" Jul 02 13:41:36.513564 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:36.513545 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 13:41:36.513662 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:36.513572 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 13:41:36.513662 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:36.513584 2561 projected.go:196] Error preparing data for projected volume kube-api-access-9gsz6 for pod openshift-network-diagnostics/network-check-target-jv42p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:41:36.513662 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:36.513645 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6 podName:684ebf83-fb83-47a1-94c7-ecd7bba444c7 nodeName:}" failed. No retries permitted until 2026-07-02 13:41:37.013629799 +0000 UTC m=+3.139766299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9gsz6" (UniqueName: "kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6") pod "network-check-target-jv42p" (UID: "684ebf83-fb83-47a1-94c7-ecd7bba444c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:41:36.518250 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.518227 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wmzl\" (UniqueName: \"kubernetes.io/projected/4d160c6f-4496-4913-a38a-26f61cd3effe-kube-api-access-7wmzl\") pod \"node-resolver-kcdlm\" (UID: \"4d160c6f-4496-4913-a38a-26f61cd3effe\") " pod="openshift-dns/node-resolver-kcdlm" Jul 02 13:41:36.518250 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.518236 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnn4c\" (UniqueName: \"kubernetes.io/projected/5a0abd99-eab7-4570-bfee-9c54ac664871-kube-api-access-hnn4c\") pod \"aws-ebs-csi-driver-node-7dwf4\" (UID: \"5a0abd99-eab7-4570-bfee-9c54ac664871\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.518671 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.518647 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fckz5\" (UniqueName: \"kubernetes.io/projected/bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea-kube-api-access-fckz5\") pod \"multus-xdsnn\" (UID: \"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea\") " pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.519815 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.519789 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-498vp\" (UniqueName: \"kubernetes.io/projected/c28ced17-5ab2-4167-944a-91104427887e-kube-api-access-498vp\") pod \"tuned-x5pxb\" (UID: \"c28ced17-5ab2-4167-944a-91104427887e\") " pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.521213 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.521169 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxnv4\" (UniqueName: \"kubernetes.io/projected/1aec8ab3-45e7-422c-985b-0f5541ed6149-kube-api-access-wxnv4\") pod \"node-ca-cpjcf\" (UID: \"1aec8ab3-45e7-422c-985b-0f5541ed6149\") " pod="openshift-image-registry/node-ca-cpjcf" Jul 02 13:41:36.521213 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.521185 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq8hh\" (UniqueName: \"kubernetes.io/projected/da0298d6-5c20-459a-864b-0a3146e256ee-kube-api-access-hq8hh\") pod \"network-metrics-daemon-j9rh2\" (UID: \"da0298d6-5c20-459a-864b-0a3146e256ee\") " pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:36.601175 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.601088 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p66lc" Jul 02 13:41:36.608813 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.608789 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hz9sp" Jul 02 13:41:36.611061 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:36.611033 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ac92ae9_c78b_4c00_ae36_41e1c2f5e692.slice/crio-2f50cf5b4f766fccc713fde6b9385b5613fcb5f44c680fb858046828be3c61f8 WatchSource:0}: Error finding container 2f50cf5b4f766fccc713fde6b9385b5613fcb5f44c680fb858046828be3c61f8: Status 404 returned error can't find the container with id 2f50cf5b4f766fccc713fde6b9385b5613fcb5f44c680fb858046828be3c61f8 Jul 02 13:41:36.619710 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.619588 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:41:36.620537 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:36.620501 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5b42e80_1152_44f6_8d39_fca751ae0e46.slice/crio-446e21dc89d84335fc457f77c40462c9789b74597dc6b7ff48b8e934bb4b2604 WatchSource:0}: Error finding container 446e21dc89d84335fc457f77c40462c9789b74597dc6b7ff48b8e934bb4b2604: Status 404 returned error can't find the container with id 446e21dc89d84335fc457f77c40462c9789b74597dc6b7ff48b8e934bb4b2604 Jul 02 13:41:36.627829 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.627811 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cpjcf" Jul 02 13:41:36.628730 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:36.628708 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddf20f35_9cd8_406c_b529_8e2949a8d4e5.slice/crio-1e8d00d87b10bb35b29bc099c9a613c52c6a6c5bb13e2d96789413147e00c326 WatchSource:0}: Error finding container 1e8d00d87b10bb35b29bc099c9a613c52c6a6c5bb13e2d96789413147e00c326: Status 404 returned error can't find the container with id 1e8d00d87b10bb35b29bc099c9a613c52c6a6c5bb13e2d96789413147e00c326 Jul 02 13:41:36.636137 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.636116 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" Jul 02 13:41:36.643295 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.643273 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-xdsnn" Jul 02 13:41:36.651068 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.651043 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kcdlm" Jul 02 13:41:36.658691 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.658669 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" Jul 02 13:41:36.665352 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.665331 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gp6w6" Jul 02 13:41:36.702396 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.702368 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 13:41:36.912567 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:36.912532 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret\") pod \"global-pull-secret-syncer-flv4b\" (UID: \"cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f\") " pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:36.912717 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:36.912669 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 13:41:36.912769 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:36.912737 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret podName:cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f nodeName:}" failed. No retries permitted until 2026-07-02 13:41:37.91272047 +0000 UTC m=+4.038856953 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret") pod "global-pull-secret-syncer-flv4b" (UID: "cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f") : object "kube-system"/"original-pull-secret" not registered Jul 02 13:41:37.013019 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:37.012986 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs\") pod \"network-metrics-daemon-j9rh2\" (UID: \"da0298d6-5c20-459a-864b-0a3146e256ee\") " pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:37.013165 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:37.013090 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:41:37.013165 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:37.013159 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs podName:da0298d6-5c20-459a-864b-0a3146e256ee nodeName:}" failed. No retries permitted until 2026-07-02 13:41:38.013145145 +0000 UTC m=+4.139281629 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs") pod "network-metrics-daemon-j9rh2" (UID: "da0298d6-5c20-459a-864b-0a3146e256ee") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:41:37.113893 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:37.113544 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gsz6\" (UniqueName: \"kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6\") pod \"network-check-target-jv42p\" (UID: \"684ebf83-fb83-47a1-94c7-ecd7bba444c7\") " pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:41:37.113893 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:37.113716 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 13:41:37.113893 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:37.113734 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 13:41:37.113893 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:37.113747 2561 projected.go:196] Error preparing data for projected volume kube-api-access-9gsz6 for pod openshift-network-diagnostics/network-check-target-jv42p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:41:37.113893 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:37.113837 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6 podName:684ebf83-fb83-47a1-94c7-ecd7bba444c7 nodeName:}" failed. No retries permitted until 2026-07-02 13:41:38.113816088 +0000 UTC m=+4.239952575 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gsz6" (UniqueName: "kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6") pod "network-check-target-jv42p" (UID: "684ebf83-fb83-47a1-94c7-ecd7bba444c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:41:37.331321 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:37.331279 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 13:36:36 +0000 UTC" deadline="2028-01-18 17:43:09.651283527 +0000 UTC" Jul 02 13:41:37.331321 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:37.331317 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13564h1m32.31997128s" Jul 02 13:41:37.358470 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:37.357923 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:37.358470 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:37.358076 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:41:37.367190 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:37.367016 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" event={"ID":"ddf20f35-9cd8-406c-b529-8e2949a8d4e5","Type":"ContainerStarted","Data":"1e8d00d87b10bb35b29bc099c9a613c52c6a6c5bb13e2d96789413147e00c326"} Jul 02 13:41:37.368988 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:37.368941 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hz9sp" event={"ID":"a5b42e80-1152-44f6-8d39-fca751ae0e46","Type":"ContainerStarted","Data":"446e21dc89d84335fc457f77c40462c9789b74597dc6b7ff48b8e934bb4b2604"} Jul 02 13:41:37.370696 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:37.370663 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p66lc" event={"ID":"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692","Type":"ContainerStarted","Data":"2f50cf5b4f766fccc713fde6b9385b5613fcb5f44c680fb858046828be3c61f8"} Jul 02 13:41:37.373474 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:37.373443 2561 generic.go:356] "Generic (PLEG): container finished" podID="f61c5e3d2a1741508891516fc764b65e" containerID="f12e27cdce3b4f30c3019505334214a90961a54415d30616efed7f4ca95752ba" exitCode=0 Jul 02 13:41:37.373609 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:37.373484 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-112.ec2.internal" event={"ID":"f61c5e3d2a1741508891516fc764b65e","Type":"ContainerDied","Data":"f12e27cdce3b4f30c3019505334214a90961a54415d30616efed7f4ca95752ba"} Jul 02 13:41:37.413394 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:37.413346 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1aec8ab3_45e7_422c_985b_0f5541ed6149.slice/crio-4c87594a441559ea2c3de904e132a1180fdca84c88d6880cd01fbef31ce8d21c WatchSource:0}: Error finding container 4c87594a441559ea2c3de904e132a1180fdca84c88d6880cd01fbef31ce8d21c: Status 404 returned error can't find the container with id 4c87594a441559ea2c3de904e132a1180fdca84c88d6880cd01fbef31ce8d21c Jul 02 13:41:37.424671 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:37.424317 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a0abd99_eab7_4570_bfee_9c54ac664871.slice/crio-5bf238652aab134513669d53fe7eff665f5b8fbda4b6a09cc7817126944136de WatchSource:0}: Error finding container 5bf238652aab134513669d53fe7eff665f5b8fbda4b6a09cc7817126944136de: Status 404 returned error can't find the container with id 5bf238652aab134513669d53fe7eff665f5b8fbda4b6a09cc7817126944136de Jul 02 13:41:37.427875 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:37.427766 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbea4fbcc_4063_4f71_a0e5_1e6a7ae875ea.slice/crio-5a01fe61e5b15f56cdf1c41959d39db3975d4bea2d4a52281e3e538a7ee4db01 WatchSource:0}: Error finding container 5a01fe61e5b15f56cdf1c41959d39db3975d4bea2d4a52281e3e538a7ee4db01: Status 404 returned error can't find the container with id 5a01fe61e5b15f56cdf1c41959d39db3975d4bea2d4a52281e3e538a7ee4db01 Jul 02 13:41:37.452887 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:41:37.452861 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf8c5897_0051_41d8_bc25_2f990a7dd74b.slice/crio-4637c272b06fb16e4cda730cd6fe7abf3fe2f1ae2404fb7a53dff052533ce0aa WatchSource:0}: Error finding container 4637c272b06fb16e4cda730cd6fe7abf3fe2f1ae2404fb7a53dff052533ce0aa: Status 404 returned error can't find the container with id 4637c272b06fb16e4cda730cd6fe7abf3fe2f1ae2404fb7a53dff052533ce0aa Jul 02 13:41:37.921750 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:37.921035 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret\") pod \"global-pull-secret-syncer-flv4b\" (UID: \"cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f\") " pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:37.921750 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:37.921212 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 13:41:37.921750 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:37.921276 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret podName:cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f nodeName:}" failed. No retries permitted until 2026-07-02 13:41:39.92125596 +0000 UTC m=+6.047392464 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret") pod "global-pull-secret-syncer-flv4b" (UID: "cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f") : object "kube-system"/"original-pull-secret" not registered Jul 02 13:41:38.022534 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:38.022442 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs\") pod \"network-metrics-daemon-j9rh2\" (UID: \"da0298d6-5c20-459a-864b-0a3146e256ee\") " pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:38.022698 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:38.022649 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:41:38.022757 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:38.022714 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs podName:da0298d6-5c20-459a-864b-0a3146e256ee nodeName:}" failed. No retries permitted until 2026-07-02 13:41:40.022695462 +0000 UTC m=+6.148831953 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs") pod "network-metrics-daemon-j9rh2" (UID: "da0298d6-5c20-459a-864b-0a3146e256ee") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:41:38.123238 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:38.123160 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gsz6\" (UniqueName: \"kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6\") pod \"network-check-target-jv42p\" (UID: \"684ebf83-fb83-47a1-94c7-ecd7bba444c7\") " pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:41:38.123467 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:38.123361 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 13:41:38.123467 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:38.123384 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 13:41:38.123467 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:38.123399 2561 projected.go:196] Error preparing data for projected volume kube-api-access-9gsz6 for pod openshift-network-diagnostics/network-check-target-jv42p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:41:38.123467 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:38.123458 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6 podName:684ebf83-fb83-47a1-94c7-ecd7bba444c7 nodeName:}" failed. No retries permitted until 2026-07-02 13:41:40.123439948 +0000 UTC m=+6.249576452 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gsz6" (UniqueName: "kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6") pod "network-check-target-jv42p" (UID: "684ebf83-fb83-47a1-94c7-ecd7bba444c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:41:38.332007 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:38.331869 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 13:36:36 +0000 UTC" deadline="2028-02-18 00:43:51.344593658 +0000 UTC" Jul 02 13:41:38.332007 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:38.331943 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14291h2m13.012654638s" Jul 02 13:41:38.369594 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:38.369525 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:41:38.369748 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:38.369655 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:41:38.370175 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:38.370153 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:38.370287 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:38.370267 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:41:38.383024 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:38.382368 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-112.ec2.internal" event={"ID":"8f7c1481fcbae9bd1ac61a7f8b2618a4","Type":"ContainerStarted","Data":"208c48f5d285ea0313cce51dfbc10caf16a7f9e420fe27a59927ce16235b44e5"} Jul 02 13:41:38.395757 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:38.395700 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-112.ec2.internal" podStartSLOduration=3.395681925 podStartE2EDuration="3.395681925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:41:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:41:38.395018548 +0000 UTC m=+4.521155054" watchObservedRunningTime="2026-07-02 13:41:38.395681925 +0000 UTC m=+4.521818431" Jul 02 13:41:38.397245 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:38.397213 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" event={"ID":"c28ced17-5ab2-4167-944a-91104427887e","Type":"ContainerStarted","Data":"34b51441ce60b0a8f01b73f6e79e6f286c1b786af7a5cb879a542a00b90bbfa5"} Jul 02 13:41:38.399709 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:38.399617 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xdsnn" event={"ID":"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea","Type":"ContainerStarted","Data":"5a01fe61e5b15f56cdf1c41959d39db3975d4bea2d4a52281e3e538a7ee4db01"} Jul 02 13:41:38.409474 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:38.409439 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" event={"ID":"5a0abd99-eab7-4570-bfee-9c54ac664871","Type":"ContainerStarted","Data":"5bf238652aab134513669d53fe7eff665f5b8fbda4b6a09cc7817126944136de"} Jul 02 13:41:38.421182 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:38.421105 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cpjcf" event={"ID":"1aec8ab3-45e7-422c-985b-0f5541ed6149","Type":"ContainerStarted","Data":"4c87594a441559ea2c3de904e132a1180fdca84c88d6880cd01fbef31ce8d21c"} Jul 02 13:41:38.431456 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:38.431418 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-112.ec2.internal" event={"ID":"f61c5e3d2a1741508891516fc764b65e","Type":"ContainerStarted","Data":"4da9fedd3031ceaa3b657dc15a7146a0c49688927eac095db532413506dc7088"} Jul 02 13:41:38.438761 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:38.438689 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gp6w6" event={"ID":"bf8c5897-0051-41d8-bc25-2f990a7dd74b","Type":"ContainerStarted","Data":"4637c272b06fb16e4cda730cd6fe7abf3fe2f1ae2404fb7a53dff052533ce0aa"} Jul 02 13:41:38.442259 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:38.442186 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kcdlm" event={"ID":"4d160c6f-4496-4913-a38a-26f61cd3effe","Type":"ContainerStarted","Data":"e8be169ac96e937436fb4be66eeafede14fccc135d4af8959cc75587464de1ff"} Jul 02 13:41:39.357607 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:39.357573 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:39.358099 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:39.357704 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:41:39.939295 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:39.938650 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret\") pod \"global-pull-secret-syncer-flv4b\" (UID: \"cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f\") " pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:39.939295 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:39.938818 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 13:41:39.939295 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:39.938889 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret podName:cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f nodeName:}" failed. No retries permitted until 2026-07-02 13:41:43.93886965 +0000 UTC m=+10.065006149 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret") pod "global-pull-secret-syncer-flv4b" (UID: "cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f") : object "kube-system"/"original-pull-secret" not registered Jul 02 13:41:40.039856 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:40.039817 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs\") pod \"network-metrics-daemon-j9rh2\" (UID: \"da0298d6-5c20-459a-864b-0a3146e256ee\") " pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:40.040082 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:40.040021 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:41:40.040149 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:40.040085 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs podName:da0298d6-5c20-459a-864b-0a3146e256ee nodeName:}" failed. No retries permitted until 2026-07-02 13:41:44.04006611 +0000 UTC m=+10.166202610 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs") pod "network-metrics-daemon-j9rh2" (UID: "da0298d6-5c20-459a-864b-0a3146e256ee") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:41:40.140257 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:40.140211 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gsz6\" (UniqueName: \"kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6\") pod \"network-check-target-jv42p\" (UID: \"684ebf83-fb83-47a1-94c7-ecd7bba444c7\") " pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:41:40.140499 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:40.140478 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 13:41:40.140572 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:40.140507 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 13:41:40.140572 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:40.140520 2561 projected.go:196] Error preparing data for projected volume kube-api-access-9gsz6 for pod openshift-network-diagnostics/network-check-target-jv42p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:41:40.140677 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:40.140581 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6 podName:684ebf83-fb83-47a1-94c7-ecd7bba444c7 nodeName:}" failed. No retries permitted until 2026-07-02 13:41:44.14056204 +0000 UTC m=+10.266698541 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gsz6" (UniqueName: "kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6") pod "network-check-target-jv42p" (UID: "684ebf83-fb83-47a1-94c7-ecd7bba444c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:41:40.358123 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:40.358018 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:41:40.358123 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:40.358033 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:40.358676 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:40.358180 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:41:40.358676 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:40.358239 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:41:41.358165 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:41.358128 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:41.358640 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:41.358263 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:41:42.358745 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:42.358701 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:42.359250 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:42.358858 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:41:42.359250 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:42.359084 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:41:42.359250 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:42.359198 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:41:43.357517 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:43.357489 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:43.357679 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:43.357612 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:41:43.973333 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:43.973219 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret\") pod \"global-pull-secret-syncer-flv4b\" (UID: \"cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f\") " pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:43.973797 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:43.973424 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 13:41:43.973797 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:43.973500 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret podName:cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f nodeName:}" failed. No retries permitted until 2026-07-02 13:41:51.973480046 +0000 UTC m=+18.099616532 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret") pod "global-pull-secret-syncer-flv4b" (UID: "cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f") : object "kube-system"/"original-pull-secret" not registered Jul 02 13:41:44.074555 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:44.074513 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs\") pod \"network-metrics-daemon-j9rh2\" (UID: \"da0298d6-5c20-459a-864b-0a3146e256ee\") " pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:44.074745 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:44.074683 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:41:44.074814 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:44.074757 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs podName:da0298d6-5c20-459a-864b-0a3146e256ee nodeName:}" failed. No retries permitted until 2026-07-02 13:41:52.074735118 +0000 UTC m=+18.200871618 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs") pod "network-metrics-daemon-j9rh2" (UID: "da0298d6-5c20-459a-864b-0a3146e256ee") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:41:44.175827 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:44.175195 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gsz6\" (UniqueName: \"kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6\") pod \"network-check-target-jv42p\" (UID: \"684ebf83-fb83-47a1-94c7-ecd7bba444c7\") " pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:41:44.175827 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:44.175360 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 13:41:44.175827 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:44.175378 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 13:41:44.175827 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:44.175391 2561 projected.go:196] Error preparing data for projected volume kube-api-access-9gsz6 for pod openshift-network-diagnostics/network-check-target-jv42p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:41:44.175827 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:44.175456 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6 podName:684ebf83-fb83-47a1-94c7-ecd7bba444c7 nodeName:}" failed. No retries permitted until 2026-07-02 13:41:52.175433519 +0000 UTC m=+18.301570020 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gsz6" (UniqueName: "kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6") pod "network-check-target-jv42p" (UID: "684ebf83-fb83-47a1-94c7-ecd7bba444c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:41:44.358855 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:44.358751 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:44.359036 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:44.358908 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:41:44.359036 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:44.358999 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:41:44.359036 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:44.358947 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:41:45.357987 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:45.357873 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:45.358435 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:45.358043 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:41:46.357594 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:46.357555 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:41:46.357757 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:46.357599 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:46.357757 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:46.357685 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:41:46.357874 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:46.357781 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:41:47.358178 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:47.358141 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:47.358545 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:47.358277 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:41:48.358043 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:48.358011 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:41:48.358224 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:48.358132 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:41:48.358224 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:48.358188 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:48.358624 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:48.358313 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:41:49.357401 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:49.357368 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:49.357575 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:49.357498 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:41:50.357944 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:50.357901 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:50.357944 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:50.357932 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:41:50.358479 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:50.358057 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:41:50.358479 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:50.358188 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:41:51.358333 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:51.358247 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:51.358851 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:51.358385 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:41:52.028555 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:52.028510 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret\") pod \"global-pull-secret-syncer-flv4b\" (UID: \"cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f\") " pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:52.028737 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:52.028676 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 13:41:52.028807 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:52.028748 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret podName:cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f nodeName:}" failed. No retries permitted until 2026-07-02 13:42:08.028731431 +0000 UTC m=+34.154867920 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret") pod "global-pull-secret-syncer-flv4b" (UID: "cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f") : object "kube-system"/"original-pull-secret" not registered Jul 02 13:41:52.129653 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:52.129619 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs\") pod \"network-metrics-daemon-j9rh2\" (UID: \"da0298d6-5c20-459a-864b-0a3146e256ee\") " pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:52.129830 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:52.129777 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:41:52.129908 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:52.129839 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs podName:da0298d6-5c20-459a-864b-0a3146e256ee nodeName:}" failed. No retries permitted until 2026-07-02 13:42:08.12982086 +0000 UTC m=+34.255957355 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs") pod "network-metrics-daemon-j9rh2" (UID: "da0298d6-5c20-459a-864b-0a3146e256ee") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:41:52.230926 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:52.230891 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gsz6\" (UniqueName: \"kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6\") pod \"network-check-target-jv42p\" (UID: \"684ebf83-fb83-47a1-94c7-ecd7bba444c7\") " pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:41:52.231139 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:52.231072 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 13:41:52.231139 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:52.231092 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 13:41:52.231139 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:52.231103 2561 projected.go:196] Error preparing data for projected volume kube-api-access-9gsz6 for pod openshift-network-diagnostics/network-check-target-jv42p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:41:52.231303 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:52.231174 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6 podName:684ebf83-fb83-47a1-94c7-ecd7bba444c7 nodeName:}" failed. No retries permitted until 2026-07-02 13:42:08.23114842 +0000 UTC m=+34.357284903 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gsz6" (UniqueName: "kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6") pod "network-check-target-jv42p" (UID: "684ebf83-fb83-47a1-94c7-ecd7bba444c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:41:52.358182 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:52.358101 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:41:52.358350 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:52.358101 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:52.358350 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:52.358219 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:41:52.358350 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:52.358318 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:41:53.357824 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:53.357792 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:53.358015 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:53.357925 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:41:54.358152 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:54.358117 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:54.358613 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:54.358196 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:41:54.358613 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:54.358223 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:41:54.358613 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:54.358312 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:41:55.357705 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:55.357677 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:55.357884 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:55.357782 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:41:56.357710 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.357294 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:41:56.358504 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.357294 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:56.358504 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:56.357814 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:41:56.358504 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:56.357926 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:41:56.484043 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.484007 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-xdsnn" event={"ID":"bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea","Type":"ContainerStarted","Data":"a00c853d831878fe93497298cc8825565a9d08c2128f25e40c765dc7ff3c8426"} Jul 02 13:41:56.486369 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.486332 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" event={"ID":"5a0abd99-eab7-4570-bfee-9c54ac664871","Type":"ContainerStarted","Data":"6d27a9f4bc8fa43dc8dcbf7770b7292199c3bda21554e31e931c0d87d3d4f5be"} Jul 02 13:41:56.488691 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.488563 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cpjcf" event={"ID":"1aec8ab3-45e7-422c-985b-0f5541ed6149","Type":"ContainerStarted","Data":"923e367a0c6028bdb415deef6fb8820396cd4ae80d121756b328a9fa07aa9999"} Jul 02 13:41:56.492609 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.492576 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" event={"ID":"ddf20f35-9cd8-406c-b529-8e2949a8d4e5","Type":"ContainerStarted","Data":"7fe69d10158bd26d9d22f72a8a606c5c62598c6b2fb33e274fa3749da2ca845f"} Jul 02 13:41:56.492751 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.492666 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" event={"ID":"ddf20f35-9cd8-406c-b529-8e2949a8d4e5","Type":"ContainerStarted","Data":"d6df1e4208825bbbb23e8d517f8dd406185f903e5743ef500ca5362da937ef79"} Jul 02 13:41:56.492751 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.492719 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" event={"ID":"ddf20f35-9cd8-406c-b529-8e2949a8d4e5","Type":"ContainerStarted","Data":"042f04bac622eafbee01c956e3c6c5919fef5b6c1e20092557ef056335deed97"} Jul 02 13:41:56.492869 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.492770 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" event={"ID":"ddf20f35-9cd8-406c-b529-8e2949a8d4e5","Type":"ContainerStarted","Data":"be26b2e260dcf8e64e5bcdb30426701409c2a25adacde6db55449a514a1e1a88"} Jul 02 13:41:56.492869 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.492800 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" event={"ID":"ddf20f35-9cd8-406c-b529-8e2949a8d4e5","Type":"ContainerStarted","Data":"46aa30c338d9ff7b5337138b9b009e720841b38128eafbe4d1c740af8b460f82"} Jul 02 13:41:56.492869 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.492843 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" event={"ID":"ddf20f35-9cd8-406c-b529-8e2949a8d4e5","Type":"ContainerStarted","Data":"84db4aa8a1993fc62ede219d25c3d65c43fa87899b152a040dc85f2efe0888c6"} Jul 02 13:41:56.494228 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.494197 2561 generic.go:356] "Generic (PLEG): container finished" podID="6ac92ae9-c78b-4c00-ae36-41e1c2f5e692" containerID="e7122fa87858e75ee9975725b9c06c4a7fa6fa4d2c87e510c3891ada75c61f2f" exitCode=0 Jul 02 13:41:56.494342 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.494291 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p66lc" event={"ID":"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692","Type":"ContainerDied","Data":"e7122fa87858e75ee9975725b9c06c4a7fa6fa4d2c87e510c3891ada75c61f2f"} Jul 02 13:41:56.496070 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.495956 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gp6w6" event={"ID":"bf8c5897-0051-41d8-bc25-2f990a7dd74b","Type":"ContainerStarted","Data":"c5bee6c9da39b3ab85cd26b385feaa1181c214fa2eaaa921523929db750d0733"} Jul 02 13:41:56.497991 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.497929 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kcdlm" event={"ID":"4d160c6f-4496-4913-a38a-26f61cd3effe","Type":"ContainerStarted","Data":"b4f051f99e1992adc14c413cfd976e3791593365ba4ca0b463a9572f97530754"} Jul 02 13:41:56.499677 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.499654 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" event={"ID":"c28ced17-5ab2-4167-944a-91104427887e","Type":"ContainerStarted","Data":"87d46a3f131d1e451f02a35f37ab493ee5e16a87f022e4ae7f861c5786eac6c9"} Jul 02 13:41:56.509112 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.509017 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-112.ec2.internal" podStartSLOduration=21.50900229 podStartE2EDuration="21.50900229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:41:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:41:38.448318141 +0000 UTC m=+4.574454653" watchObservedRunningTime="2026-07-02 13:41:56.50900229 +0000 UTC m=+22.635138785" Jul 02 13:41:56.509262 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.509194 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-xdsnn" podStartSLOduration=4.157345598 podStartE2EDuration="22.509184235s" totalImagesPullingTime="18.351838637s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:41:56.50886907 +0000 UTC m=+22.635005600" watchObservedRunningTime="2026-07-02 13:41:56.509184235 +0000 UTC m=+22.635320742" Jul 02 13:41:56.523908 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.523859 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cpjcf" podStartSLOduration=4.205028208 podStartE2EDuration="22.523842754s" totalImagesPullingTime="18.318814546s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:41:56.52369771 +0000 UTC m=+22.649834217" watchObservedRunningTime="2026-07-02 13:41:56.523842754 +0000 UTC m=+22.649979309" Jul 02 13:41:56.561941 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.561880 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kcdlm" podStartSLOduration=9.014611671 podStartE2EDuration="22.561865634s" totalImagesPullingTime="13.547253963s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:41:56.561678304 +0000 UTC m=+22.687814826" watchObservedRunningTime="2026-07-02 13:41:56.561865634 +0000 UTC m=+22.688002140" Jul 02 13:41:56.580288 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.580238 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-x5pxb" podStartSLOduration=4.292364751 podStartE2EDuration="22.580219596s" totalImagesPullingTime="18.287854845s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:41:56.579547816 +0000 UTC m=+22.705684324" watchObservedRunningTime="2026-07-02 13:41:56.580219596 +0000 UTC m=+22.706356117" Jul 02 13:41:56.599766 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:56.599717 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gp6w6" podStartSLOduration=4.3192953339999995 podStartE2EDuration="22.599698133s" totalImagesPullingTime="18.280402799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:41:56.599373237 +0000 UTC m=+22.725509742" watchObservedRunningTime="2026-07-02 13:41:56.599698133 +0000 UTC m=+22.725834640" Jul 02 13:41:57.231534 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:57.231501 2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 02 13:41:57.357851 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:57.357771 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:57.358309 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:57.357907 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:41:57.379237 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:57.379096 2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T13:41:57.231522778Z","UUID":"6bbbe45b-47b3-4ad1-b108-9172d730f2dd","Handler":null,"Name":"","Endpoint":""} Jul 02 13:41:57.380826 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:57.380805 2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 02 13:41:57.380941 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:57.380845 2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 02 13:41:57.504032 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:57.503993 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hz9sp" event={"ID":"a5b42e80-1152-44f6-8d39-fca751ae0e46","Type":"ContainerStarted","Data":"dc40af7e7e5e0ebb8a8a24fa62ad43a0101d4208ee2f3ad275d103c35b0813b4"} Jul 02 13:41:57.506135 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:57.506098 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" event={"ID":"5a0abd99-eab7-4570-bfee-9c54ac664871","Type":"ContainerStarted","Data":"92c6eda4d77d719f39395afe95e8a6ec3d288bfc8f0e75584ef6acdac01a1372"} Jul 02 13:41:57.518481 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:57.518431 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hz9sp" podStartSLOduration=9.143533466 podStartE2EDuration="23.518411689s" totalImagesPullingTime="14.374878223s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:41:57.517800691 +0000 UTC m=+23.643937198" watchObservedRunningTime="2026-07-02 13:41:57.518411689 +0000 UTC m=+23.644548198" Jul 02 13:41:58.358586 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:58.358276 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:41:58.358586 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:58.358286 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:41:58.358586 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:58.358403 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:41:58.358586 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:58.358529 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:41:58.510807 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:58.510765 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" event={"ID":"5a0abd99-eab7-4570-bfee-9c54ac664871","Type":"ContainerStarted","Data":"3ebcb3f570c37c3c42bb2521475706ba3b89001260454da59bee2eecb6934e26"} Jul 02 13:41:58.529952 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:58.529888 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7dwf4" podStartSLOduration=3.957377553 podStartE2EDuration="24.529868008s" totalImagesPullingTime="20.572490455s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:41:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:41:58.529562771 +0000 UTC m=+24.655699276" watchObservedRunningTime="2026-07-02 13:41:58.529868008 +0000 UTC m=+24.656004515" Jul 02 13:41:59.357922 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:59.357889 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:41:59.358169 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:41:59.358032 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:41:59.405156 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:59.405122 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gp6w6" Jul 02 13:41:59.405664 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:59.405645 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gp6w6" Jul 02 13:41:59.517805 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:59.517764 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" event={"ID":"ddf20f35-9cd8-406c-b529-8e2949a8d4e5","Type":"ContainerStarted","Data":"a65016ce0a3af2f496ca4c155dc5262b9eb8c9b60d1ceb82c2dc9c813343cec4"} Jul 02 13:41:59.518037 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:59.518015 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gp6w6" Jul 02 13:41:59.518576 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:41:59.518557 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gp6w6" Jul 02 13:42:00.358285 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:00.358059 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:42:00.358477 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:00.358122 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:42:00.358477 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:00.358396 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:42:00.358477 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:00.358463 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:42:01.358078 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:01.357882 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:42:01.358682 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:01.358167 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:42:01.526371 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:01.526258 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" event={"ID":"ddf20f35-9cd8-406c-b529-8e2949a8d4e5","Type":"ContainerStarted","Data":"c6ea2e50dff270aa3c77bd2e10643831586d9ba7199bd22d4c9d5a20d593555b"} Jul 02 13:42:01.526555 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:01.526535 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:42:01.528037 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:01.528013 2561 generic.go:356] "Generic (PLEG): container finished" podID="6ac92ae9-c78b-4c00-ae36-41e1c2f5e692" containerID="4d6ff7564ca15a3d94b9c51a2409755e960fd31077f92917f26834d100fe4875" exitCode=0 Jul 02 13:42:01.528138 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:01.528094 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p66lc" event={"ID":"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692","Type":"ContainerDied","Data":"4d6ff7564ca15a3d94b9c51a2409755e960fd31077f92917f26834d100fe4875"} Jul 02 13:42:01.542271 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:01.542252 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:42:01.553832 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:01.553774 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" podStartSLOduration=8.374253241 podStartE2EDuration="27.553759155s" totalImagesPullingTime="19.179505914s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:42:01.553643344 +0000 UTC m=+27.679779849" watchObservedRunningTime="2026-07-02 13:42:01.553759155 +0000 UTC m=+27.679895659" Jul 02 13:42:02.358252 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:02.358171 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:42:02.358690 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:02.358175 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:42:02.358690 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:02.358314 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:42:02.358690 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:02.358379 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:42:02.530771 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:02.530725 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:42:02.530946 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:02.530820 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:42:02.546317 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:02.546293 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:42:03.358070 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:03.358033 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:42:03.358262 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:03.358185 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:42:03.432524 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:03.432487 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-flv4b"] Jul 02 13:42:03.433298 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:03.433274 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jv42p"] Jul 02 13:42:03.433448 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:03.433375 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:42:03.433510 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:03.433469 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:42:03.433862 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:03.433838 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j9rh2"] Jul 02 13:42:03.433944 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:03.433933 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:42:03.434071 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:03.434052 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:42:03.535154 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:03.535056 2561 generic.go:356] "Generic (PLEG): container finished" podID="6ac92ae9-c78b-4c00-ae36-41e1c2f5e692" containerID="69faffcd625b9db9957ac41e19d5f7723c307af5776a1f493865c56325fb2d18" exitCode=0 Jul 02 13:42:03.535154 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:03.535128 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:42:03.535154 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:03.535136 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p66lc" event={"ID":"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692","Type":"ContainerDied","Data":"69faffcd625b9db9957ac41e19d5f7723c307af5776a1f493865c56325fb2d18"} Jul 02 13:42:03.535385 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:03.535362 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:42:05.358117 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:05.357874 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:42:05.358606 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:05.358235 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:42:05.360173 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:05.358681 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:42:05.360173 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:05.358800 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:42:05.360173 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:05.358963 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:42:05.360173 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:05.359082 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:42:05.542398 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:05.542362 2561 generic.go:356] "Generic (PLEG): container finished" podID="6ac92ae9-c78b-4c00-ae36-41e1c2f5e692" containerID="6d679b2cfab19afeb348c565db4107544d383688feafe88d53cf8d9e8321733f" exitCode=0 Jul 02 13:42:05.542551 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:05.542408 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p66lc" event={"ID":"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692","Type":"ContainerDied","Data":"6d679b2cfab19afeb348c565db4107544d383688feafe88d53cf8d9e8321733f"} Jul 02 13:42:07.358122 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:07.358095 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:42:07.358122 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:07.358096 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:42:07.358658 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:07.358096 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:42:07.358658 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:07.358205 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:42:07.358658 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:07.358291 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:42:07.358658 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:07.358360 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:42:08.047180 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:08.047129 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret\") pod \"global-pull-secret-syncer-flv4b\" (UID: \"cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f\") " pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:42:08.047368 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:08.047339 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 13:42:08.047441 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:08.047415 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret podName:cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f nodeName:}" failed. No retries permitted until 2026-07-02 13:42:40.047392367 +0000 UTC m=+66.173528861 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret") pod "global-pull-secret-syncer-flv4b" (UID: "cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f") : object "kube-system"/"original-pull-secret" not registered Jul 02 13:42:08.148176 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:08.148124 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs\") pod \"network-metrics-daemon-j9rh2\" (UID: \"da0298d6-5c20-459a-864b-0a3146e256ee\") " pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:42:08.148330 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:08.148241 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:42:08.148330 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:08.148313 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs podName:da0298d6-5c20-459a-864b-0a3146e256ee nodeName:}" failed. No retries permitted until 2026-07-02 13:42:40.148294114 +0000 UTC m=+66.274430604 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs") pod "network-metrics-daemon-j9rh2" (UID: "da0298d6-5c20-459a-864b-0a3146e256ee") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 13:42:08.249348 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:08.249310 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gsz6\" (UniqueName: \"kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6\") pod \"network-check-target-jv42p\" (UID: \"684ebf83-fb83-47a1-94c7-ecd7bba444c7\") " pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:42:08.249532 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:08.249495 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 13:42:08.249532 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:08.249518 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 13:42:08.249532 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:08.249531 2561 projected.go:196] Error preparing data for projected volume kube-api-access-9gsz6 for pod openshift-network-diagnostics/network-check-target-jv42p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:42:08.249693 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:08.249601 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6 podName:684ebf83-fb83-47a1-94c7-ecd7bba444c7 nodeName:}" failed. No retries permitted until 2026-07-02 13:42:40.249578981 +0000 UTC m=+66.375715485 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9gsz6" (UniqueName: "kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6") pod "network-check-target-jv42p" (UID: "684ebf83-fb83-47a1-94c7-ecd7bba444c7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 13:42:09.357796 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:09.357758 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:42:09.358424 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:09.357758 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:42:09.358424 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:09.357898 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:42:09.358424 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:09.357758 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:42:09.358424 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:09.358029 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:42:09.358424 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:09.358067 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:42:10.556128 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:10.556088 2561 generic.go:356] "Generic (PLEG): container finished" podID="6ac92ae9-c78b-4c00-ae36-41e1c2f5e692" containerID="b001d1269cc16719a7381185ac6de7f959fa0f504d6afbbedc3731bcf780bcfb" exitCode=0 Jul 02 13:42:10.556575 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:10.556138 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p66lc" event={"ID":"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692","Type":"ContainerDied","Data":"b001d1269cc16719a7381185ac6de7f959fa0f504d6afbbedc3731bcf780bcfb"} Jul 02 13:42:11.358262 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:11.358213 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:42:11.358461 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:11.358334 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:42:11.358461 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:11.358226 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:42:11.358461 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:11.358241 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:42:11.358461 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:11.358445 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:42:11.358594 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:11.358513 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:42:11.561850 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:11.561822 2561 generic.go:356] "Generic (PLEG): container finished" podID="6ac92ae9-c78b-4c00-ae36-41e1c2f5e692" containerID="8da247a08cbaa3c584f2604d1c056f14888c447bcdb43d0a53e49a7c39ddf884" exitCode=0 Jul 02 13:42:11.562293 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:11.561891 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p66lc" event={"ID":"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692","Type":"ContainerDied","Data":"8da247a08cbaa3c584f2604d1c056f14888c447bcdb43d0a53e49a7c39ddf884"} Jul 02 13:42:12.567411 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:12.567379 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p66lc" event={"ID":"6ac92ae9-c78b-4c00-ae36-41e1c2f5e692","Type":"ContainerStarted","Data":"e56a8ab65e454774640f9f36045cdf4bc767ee991a98f6d0b25e36a3773e66de"} Jul 02 13:42:12.593226 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:12.593179 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-p66lc" podStartSLOduration=7.936464997 podStartE2EDuration="38.593164512s" totalImagesPullingTime="30.656699515s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:41:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:42:12.592712537 +0000 UTC m=+38.718849054" watchObservedRunningTime="2026-07-02 13:42:12.593164512 +0000 UTC m=+38.719301017" Jul 02 13:42:13.357415 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:13.357231 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:42:13.357568 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:13.357230 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:42:13.357568 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:13.357492 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:42:13.357568 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:13.357233 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:42:13.357568 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:13.357555 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:42:13.357733 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:13.357611 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:42:15.357443 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:15.357398 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:42:15.357875 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:15.357398 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:42:15.357875 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:15.357516 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:42:15.357875 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:15.357405 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:42:15.357875 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:15.357589 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:42:15.357875 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:15.357676 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:42:17.358092 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:17.358057 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:42:17.358470 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:17.358057 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:42:17.358470 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:17.358173 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:42:17.358470 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:17.358057 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:42:17.358470 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:17.358237 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:42:17.358470 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:17.358303 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:42:19.358221 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:19.358190 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:42:19.358221 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:19.358216 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:42:19.358717 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:19.358190 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:42:19.358717 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:19.358292 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:42:19.358717 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:19.358374 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:42:19.358717 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:19.358426 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:42:21.358242 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:21.358196 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:42:21.358733 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:21.358212 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:42:21.358733 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:21.358317 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:42:21.358733 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:21.358328 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:42:21.358733 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:21.358461 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:42:21.358733 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:21.358529 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:42:23.357257 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:23.357219 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:42:23.357721 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:23.357225 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:42:23.357721 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:23.357343 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:42:23.357721 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:23.357225 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:42:23.357721 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:23.357413 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:42:23.357721 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:23.357490 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:42:25.357992 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:25.357934 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:42:25.358378 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:25.358004 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:42:25.358378 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:25.358059 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:42:25.358378 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:25.358077 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:42:25.358378 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:25.358138 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:42:25.358378 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:25.358203 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:42:27.358093 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:27.358063 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:42:27.358093 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:27.358086 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:42:27.358579 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:27.358062 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:42:27.358579 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:27.358163 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-j9rh2" podUID="da0298d6-5c20-459a-864b-0a3146e256ee" Jul 02 13:42:27.358579 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:27.358221 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jv42p" podUID="684ebf83-fb83-47a1-94c7-ecd7bba444c7" Jul 02 13:42:27.358579 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:27.358275 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-flv4b" podUID="cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f" Jul 02 13:42:28.655410 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.655329 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-112.ec2.internal" event="NodeReady" Jul 02 13:42:28.655771 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.655468 2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 02 13:42:28.729611 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.729580 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dwb5c"] Jul 02 13:42:28.731431 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.731416 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dwb5c" Jul 02 13:42:28.733955 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.733922 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 02 13:42:28.733955 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.733952 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 02 13:42:28.734179 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.733953 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jsckw\"" Jul 02 13:42:28.734179 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.733952 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 02 13:42:28.755265 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.755242 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dwb5c"] Jul 02 13:42:28.840941 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.840908 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dwvk4"] Jul 02 13:42:28.842841 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.842819 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dwvk4" Jul 02 13:42:28.845320 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.845290 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 02 13:42:28.845650 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.845631 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 02 13:42:28.845708 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.845666 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6mf97\"" Jul 02 13:42:28.863535 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.863502 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dwvk4"] Jul 02 13:42:28.895191 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.895148 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjtns\" (UniqueName: \"kubernetes.io/projected/a509cdb7-1c6e-4e40-9add-d802b749c4b0-kube-api-access-mjtns\") pod \"ingress-canary-dwb5c\" (UID: \"a509cdb7-1c6e-4e40-9add-d802b749c4b0\") " pod="openshift-ingress-canary/ingress-canary-dwb5c" Jul 02 13:42:28.895191 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.895204 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a509cdb7-1c6e-4e40-9add-d802b749c4b0-cert\") pod \"ingress-canary-dwb5c\" (UID: \"a509cdb7-1c6e-4e40-9add-d802b749c4b0\") " pod="openshift-ingress-canary/ingress-canary-dwb5c" Jul 02 13:42:28.996134 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.996104 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/db8817af-4159-4667-b285-8e6325a345c4-metrics-tls\") pod \"dns-default-dwvk4\" (UID: \"db8817af-4159-4667-b285-8e6325a345c4\") " pod="openshift-dns/dns-default-dwvk4" Jul 02 13:42:28.996298 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.996154 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjtns\" (UniqueName: \"kubernetes.io/projected/a509cdb7-1c6e-4e40-9add-d802b749c4b0-kube-api-access-mjtns\") pod \"ingress-canary-dwb5c\" (UID: \"a509cdb7-1c6e-4e40-9add-d802b749c4b0\") " pod="openshift-ingress-canary/ingress-canary-dwb5c" Jul 02 13:42:28.996298 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.996187 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wtbn\" (UniqueName: \"kubernetes.io/projected/db8817af-4159-4667-b285-8e6325a345c4-kube-api-access-4wtbn\") pod \"dns-default-dwvk4\" (UID: \"db8817af-4159-4667-b285-8e6325a345c4\") " pod="openshift-dns/dns-default-dwvk4" Jul 02 13:42:28.996298 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.996204 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a509cdb7-1c6e-4e40-9add-d802b749c4b0-cert\") pod \"ingress-canary-dwb5c\" (UID: \"a509cdb7-1c6e-4e40-9add-d802b749c4b0\") " pod="openshift-ingress-canary/ingress-canary-dwb5c" Jul 02 13:42:28.996298 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.996218 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/db8817af-4159-4667-b285-8e6325a345c4-config-volume\") pod \"dns-default-dwvk4\" (UID: \"db8817af-4159-4667-b285-8e6325a345c4\") " pod="openshift-dns/dns-default-dwvk4" Jul 02 13:42:28.996497 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:28.996341 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db8817af-4159-4667-b285-8e6325a345c4-tmp-dir\") pod \"dns-default-dwvk4\" (UID: \"db8817af-4159-4667-b285-8e6325a345c4\") " pod="openshift-dns/dns-default-dwvk4" Jul 02 13:42:29.000586 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.000560 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a509cdb7-1c6e-4e40-9add-d802b749c4b0-cert\") pod \"ingress-canary-dwb5c\" (UID: \"a509cdb7-1c6e-4e40-9add-d802b749c4b0\") " pod="openshift-ingress-canary/ingress-canary-dwb5c" Jul 02 13:42:29.004343 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.004314 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjtns\" (UniqueName: \"kubernetes.io/projected/a509cdb7-1c6e-4e40-9add-d802b749c4b0-kube-api-access-mjtns\") pod \"ingress-canary-dwb5c\" (UID: \"a509cdb7-1c6e-4e40-9add-d802b749c4b0\") " pod="openshift-ingress-canary/ingress-canary-dwb5c" Jul 02 13:42:29.041092 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.041068 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dwb5c" Jul 02 13:42:29.097300 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.097077 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/db8817af-4159-4667-b285-8e6325a345c4-metrics-tls\") pod \"dns-default-dwvk4\" (UID: \"db8817af-4159-4667-b285-8e6325a345c4\") " pod="openshift-dns/dns-default-dwvk4" Jul 02 13:42:29.097300 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.097172 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wtbn\" (UniqueName: \"kubernetes.io/projected/db8817af-4159-4667-b285-8e6325a345c4-kube-api-access-4wtbn\") pod \"dns-default-dwvk4\" (UID: \"db8817af-4159-4667-b285-8e6325a345c4\") " pod="openshift-dns/dns-default-dwvk4" Jul 02 13:42:29.097300 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.097203 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/db8817af-4159-4667-b285-8e6325a345c4-config-volume\") pod \"dns-default-dwvk4\" (UID: \"db8817af-4159-4667-b285-8e6325a345c4\") " pod="openshift-dns/dns-default-dwvk4" Jul 02 13:42:29.097300 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.097226 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db8817af-4159-4667-b285-8e6325a345c4-tmp-dir\") pod \"dns-default-dwvk4\" (UID: \"db8817af-4159-4667-b285-8e6325a345c4\") " pod="openshift-dns/dns-default-dwvk4" Jul 02 13:42:29.098089 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.097576 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db8817af-4159-4667-b285-8e6325a345c4-tmp-dir\") pod \"dns-default-dwvk4\" (UID: \"db8817af-4159-4667-b285-8e6325a345c4\") " pod="openshift-dns/dns-default-dwvk4" Jul 02 13:42:29.098348 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.098174 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/db8817af-4159-4667-b285-8e6325a345c4-config-volume\") pod \"dns-default-dwvk4\" (UID: \"db8817af-4159-4667-b285-8e6325a345c4\") " pod="openshift-dns/dns-default-dwvk4" Jul 02 13:42:29.099880 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.099856 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/db8817af-4159-4667-b285-8e6325a345c4-metrics-tls\") pod \"dns-default-dwvk4\" (UID: \"db8817af-4159-4667-b285-8e6325a345c4\") " pod="openshift-dns/dns-default-dwvk4" Jul 02 13:42:29.106523 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.106487 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wtbn\" (UniqueName: \"kubernetes.io/projected/db8817af-4159-4667-b285-8e6325a345c4-kube-api-access-4wtbn\") pod \"dns-default-dwvk4\" (UID: \"db8817af-4159-4667-b285-8e6325a345c4\") " pod="openshift-dns/dns-default-dwvk4" Jul 02 13:42:29.151292 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.151261 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dwvk4" Jul 02 13:42:29.176940 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.176907 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dwb5c"] Jul 02 13:42:29.180516 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:42:29.180486 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda509cdb7_1c6e_4e40_9add_d802b749c4b0.slice/crio-a6f0a47860711a801db10867f13d6ca44943ea29bb400f3e668340039a0cec21 WatchSource:0}: Error finding container a6f0a47860711a801db10867f13d6ca44943ea29bb400f3e668340039a0cec21: Status 404 returned error can't find the container with id a6f0a47860711a801db10867f13d6ca44943ea29bb400f3e668340039a0cec21 Jul 02 13:42:29.275239 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.275151 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dwvk4"] Jul 02 13:42:29.278437 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:42:29.278405 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb8817af_4159_4667_b285_8e6325a345c4.slice/crio-684e195b0325750cbd7e4718c9c0ba67201cd852ab081f3f2f611dc77e205a6b WatchSource:0}: Error finding container 684e195b0325750cbd7e4718c9c0ba67201cd852ab081f3f2f611dc77e205a6b: Status 404 returned error can't find the container with id 684e195b0325750cbd7e4718c9c0ba67201cd852ab081f3f2f611dc77e205a6b Jul 02 13:42:29.357935 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.357899 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:42:29.358130 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.357899 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:42:29.358130 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.357899 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:42:29.360553 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.360517 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 13:42:29.360553 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.360542 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 13:42:29.361058 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.361040 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bhzn5\"" Jul 02 13:42:29.361167 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.361084 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tz79z\"" Jul 02 13:42:29.361412 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.361395 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 13:42:29.361502 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.361402 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 13:42:29.602586 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.602488 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dwvk4" event={"ID":"db8817af-4159-4667-b285-8e6325a345c4","Type":"ContainerStarted","Data":"684e195b0325750cbd7e4718c9c0ba67201cd852ab081f3f2f611dc77e205a6b"} Jul 02 13:42:29.603581 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:29.603559 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dwb5c" event={"ID":"a509cdb7-1c6e-4e40-9add-d802b749c4b0","Type":"ContainerStarted","Data":"a6f0a47860711a801db10867f13d6ca44943ea29bb400f3e668340039a0cec21"} Jul 02 13:42:31.610789 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:31.610749 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dwvk4" event={"ID":"db8817af-4159-4667-b285-8e6325a345c4","Type":"ContainerStarted","Data":"ce72e87792fc61d4f6037b5a568558d16abd3f78fe77af7ce4460a9ccdd6386a"} Jul 02 13:42:32.508433 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:32.508399 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-kd72q"] Jul 02 13:42:32.510161 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:32.510146 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-kd72q" Jul 02 13:42:32.512746 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:32.512724 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-htw8n\"" Jul 02 13:42:32.512937 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:32.512925 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 02 13:42:32.520232 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:32.520212 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-kd72q"] Jul 02 13:42:32.615008 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:32.614958 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dwvk4" event={"ID":"db8817af-4159-4667-b285-8e6325a345c4","Type":"ContainerStarted","Data":"fe516e502855d80df9c08222a94aa91e4485eed3f9efd5fa98d76eb007c36de0"} Jul 02 13:42:32.615435 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:32.615051 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dwvk4" Jul 02 13:42:32.616440 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:32.616420 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dwb5c" event={"ID":"a509cdb7-1c6e-4e40-9add-d802b749c4b0","Type":"ContainerStarted","Data":"ec2636dbc60c5a3ba60258cc937bd5e36b79e23bc9541aa9ddf69c8eb13b6f6d"} Jul 02 13:42:32.625986 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:32.625950 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c23c15a1-a7a9-4493-b417-f7b099854dc7-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-kd72q\" (UID: \"c23c15a1-a7a9-4493-b417-f7b099854dc7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-kd72q" Jul 02 13:42:32.638022 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:32.637941 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dwvk4" podStartSLOduration=3.003801309 podStartE2EDuration="4.637924234s" totalImagesPullingTime="1.634122925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:42:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:42:32.636983718 +0000 UTC m=+58.763120223" watchObservedRunningTime="2026-07-02 13:42:32.637924234 +0000 UTC m=+58.764060717" Jul 02 13:42:32.653550 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:32.653500 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dwb5c" podStartSLOduration=2.085255971 podStartE2EDuration="4.653483207s" totalImagesPullingTime="2.568227236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:42:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:42:32.652946911 +0000 UTC m=+58.779083428" watchObservedRunningTime="2026-07-02 13:42:32.653483207 +0000 UTC m=+58.779619713" Jul 02 13:42:32.727277 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:32.727235 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c23c15a1-a7a9-4493-b417-f7b099854dc7-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-kd72q\" (UID: \"c23c15a1-a7a9-4493-b417-f7b099854dc7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-kd72q" Jul 02 13:42:32.729741 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:32.729713 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c23c15a1-a7a9-4493-b417-f7b099854dc7-tls-certificates\") pod \"prometheus-operator-admission-webhook-7ddc8784d4-kd72q\" (UID: \"c23c15a1-a7a9-4493-b417-f7b099854dc7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-kd72q" Jul 02 13:42:32.819092 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:32.818992 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-kd72q" Jul 02 13:42:32.954805 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:32.954776 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-kd72q"] Jul 02 13:42:32.957742 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:42:32.957715 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc23c15a1_a7a9_4493_b417_f7b099854dc7.slice/crio-0d3bec1dd1eb1e7ab04022bb0218742585fda6a2a18f35c14c6255739f30b468 WatchSource:0}: Error finding container 0d3bec1dd1eb1e7ab04022bb0218742585fda6a2a18f35c14c6255739f30b468: Status 404 returned error can't find the container with id 0d3bec1dd1eb1e7ab04022bb0218742585fda6a2a18f35c14c6255739f30b468 Jul 02 13:42:33.620734 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:33.620688 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-kd72q" event={"ID":"c23c15a1-a7a9-4493-b417-f7b099854dc7","Type":"ContainerStarted","Data":"0d3bec1dd1eb1e7ab04022bb0218742585fda6a2a18f35c14c6255739f30b468"} Jul 02 13:42:34.555868 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:34.555783 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ppzmg" Jul 02 13:42:34.624691 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:34.624650 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-kd72q" event={"ID":"c23c15a1-a7a9-4493-b417-f7b099854dc7","Type":"ContainerStarted","Data":"3ab3b2bfa26c05003088a6cd8f8408e98f302f80df8555ef4adcf9157d139166"} Jul 02 13:42:34.625104 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:34.624786 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-kd72q" Jul 02 13:42:34.629500 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:34.629482 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-kd72q" Jul 02 13:42:34.640850 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:34.640796 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7ddc8784d4-kd72q" podStartSLOduration=1.313821172 podStartE2EDuration="2.640778846s" totalImagesPullingTime="1.326957674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:42:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:42:34.640142485 +0000 UTC m=+60.766278992" watchObservedRunningTime="2026-07-02 13:42:34.640778846 +0000 UTC m=+60.766915352" Jul 02 13:42:35.581778 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.581739 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-sz9zz"] Jul 02 13:42:35.584047 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.584032 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-sz9zz" Jul 02 13:42:35.587985 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.587951 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 02 13:42:35.587985 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.587964 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 02 13:42:35.588132 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.588116 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 02 13:42:35.588844 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.588826 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 02 13:42:35.588892 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.588867 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-qs5lw\"" Jul 02 13:42:35.589381 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.589270 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 02 13:42:35.619242 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.619222 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-sz9zz"] Jul 02 13:42:35.750768 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.750732 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/85818307-0372-452d-b408-3b21c8fafb90-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-sz9zz\" (UID: \"85818307-0372-452d-b408-3b21c8fafb90\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-sz9zz" Jul 02 13:42:35.750768 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.750770 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqzj7\" (UniqueName: \"kubernetes.io/projected/85818307-0372-452d-b408-3b21c8fafb90-kube-api-access-dqzj7\") pod \"prometheus-operator-7b7d896497-sz9zz\" (UID: \"85818307-0372-452d-b408-3b21c8fafb90\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-sz9zz" Jul 02 13:42:35.751192 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.750841 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85818307-0372-452d-b408-3b21c8fafb90-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-sz9zz\" (UID: \"85818307-0372-452d-b408-3b21c8fafb90\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-sz9zz" Jul 02 13:42:35.751192 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.750896 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85818307-0372-452d-b408-3b21c8fafb90-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-sz9zz\" (UID: \"85818307-0372-452d-b408-3b21c8fafb90\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-sz9zz" Jul 02 13:42:35.851539 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.851433 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85818307-0372-452d-b408-3b21c8fafb90-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-sz9zz\" (UID: \"85818307-0372-452d-b408-3b21c8fafb90\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-sz9zz" Jul 02 13:42:35.851539 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.851514 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85818307-0372-452d-b408-3b21c8fafb90-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-sz9zz\" (UID: \"85818307-0372-452d-b408-3b21c8fafb90\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-sz9zz" Jul 02 13:42:35.851761 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.851567 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/85818307-0372-452d-b408-3b21c8fafb90-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-sz9zz\" (UID: \"85818307-0372-452d-b408-3b21c8fafb90\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-sz9zz" Jul 02 13:42:35.851761 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.851592 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqzj7\" (UniqueName: \"kubernetes.io/projected/85818307-0372-452d-b408-3b21c8fafb90-kube-api-access-dqzj7\") pod \"prometheus-operator-7b7d896497-sz9zz\" (UID: \"85818307-0372-452d-b408-3b21c8fafb90\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-sz9zz" Jul 02 13:42:35.852311 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.852284 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85818307-0372-452d-b408-3b21c8fafb90-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-sz9zz\" (UID: \"85818307-0372-452d-b408-3b21c8fafb90\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-sz9zz" Jul 02 13:42:35.854003 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.853964 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85818307-0372-452d-b408-3b21c8fafb90-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-sz9zz\" (UID: \"85818307-0372-452d-b408-3b21c8fafb90\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-sz9zz" Jul 02 13:42:35.854096 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.854047 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/85818307-0372-452d-b408-3b21c8fafb90-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-sz9zz\" (UID: \"85818307-0372-452d-b408-3b21c8fafb90\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-sz9zz" Jul 02 13:42:35.863468 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.863438 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqzj7\" (UniqueName: \"kubernetes.io/projected/85818307-0372-452d-b408-3b21c8fafb90-kube-api-access-dqzj7\") pod \"prometheus-operator-7b7d896497-sz9zz\" (UID: \"85818307-0372-452d-b408-3b21c8fafb90\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-sz9zz" Jul 02 13:42:35.893298 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:35.893261 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-sz9zz" Jul 02 13:42:36.022933 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.022901 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-sz9zz"] Jul 02 13:42:36.026349 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:42:36.026318 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85818307_0372_452d_b408_3b21c8fafb90.slice/crio-5c6a07542b0a15caef840e225305c70ff7e5ff9871601c43caeca8c98ec93fd8 WatchSource:0}: Error finding container 5c6a07542b0a15caef840e225305c70ff7e5ff9871601c43caeca8c98ec93fd8: Status 404 returned error can't find the container with id 5c6a07542b0a15caef840e225305c70ff7e5ff9871601c43caeca8c98ec93fd8 Jul 02 13:42:36.102309 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.102223 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5499b6fb8f-j4blc"] Jul 02 13:42:36.104528 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.104511 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5499b6fb8f-j4blc" Jul 02 13:42:36.107184 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.107156 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 02 13:42:36.107310 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.107228 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 02 13:42:36.107608 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.107593 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-9zsnf\"" Jul 02 13:42:36.117988 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.117947 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5499b6fb8f-j4blc"] Jul 02 13:42:36.153878 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.153846 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdhwk\" (UniqueName: \"kubernetes.io/projected/c0d17ca3-aa6c-4dce-a610-b7c009989a2f-kube-api-access-pdhwk\") pod \"downloads-5499b6fb8f-j4blc\" (UID: \"c0d17ca3-aa6c-4dce-a610-b7c009989a2f\") " pod="openshift-console/downloads-5499b6fb8f-j4blc" Jul 02 13:42:36.254727 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.254692 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdhwk\" (UniqueName: \"kubernetes.io/projected/c0d17ca3-aa6c-4dce-a610-b7c009989a2f-kube-api-access-pdhwk\") pod \"downloads-5499b6fb8f-j4blc\" (UID: \"c0d17ca3-aa6c-4dce-a610-b7c009989a2f\") " pod="openshift-console/downloads-5499b6fb8f-j4blc" Jul 02 13:42:36.263076 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.263057 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdhwk\" (UniqueName: \"kubernetes.io/projected/c0d17ca3-aa6c-4dce-a610-b7c009989a2f-kube-api-access-pdhwk\") pod \"downloads-5499b6fb8f-j4blc\" (UID: \"c0d17ca3-aa6c-4dce-a610-b7c009989a2f\") " pod="openshift-console/downloads-5499b6fb8f-j4blc" Jul 02 13:42:36.414092 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.414001 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5499b6fb8f-j4blc" Jul 02 13:42:36.538090 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.538054 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5499b6fb8f-j4blc"] Jul 02 13:42:36.542589 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:42:36.542563 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0d17ca3_aa6c_4dce_a610_b7c009989a2f.slice/crio-aa61ef37d9eaebf95d466dd75ea289eaf196043c48626d5118923252367f8ad6 WatchSource:0}: Error finding container aa61ef37d9eaebf95d466dd75ea289eaf196043c48626d5118923252367f8ad6: Status 404 returned error can't find the container with id aa61ef37d9eaebf95d466dd75ea289eaf196043c48626d5118923252367f8ad6 Jul 02 13:42:36.631849 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.631810 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-sz9zz" event={"ID":"85818307-0372-452d-b408-3b21c8fafb90","Type":"ContainerStarted","Data":"5c6a07542b0a15caef840e225305c70ff7e5ff9871601c43caeca8c98ec93fd8"} Jul 02 13:42:36.633132 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.633097 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5499b6fb8f-j4blc" event={"ID":"c0d17ca3-aa6c-4dce-a610-b7c009989a2f","Type":"ContainerStarted","Data":"aa61ef37d9eaebf95d466dd75ea289eaf196043c48626d5118923252367f8ad6"} Jul 02 13:42:36.917449 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.917418 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-kvknk"] Jul 02 13:42:36.919663 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.919643 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:36.922537 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.922512 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 02 13:42:36.923272 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.923251 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 02 13:42:36.923370 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.923272 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 02 13:42:36.923370 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.923300 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 02 13:42:36.923370 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.923314 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pbpg7\"" Jul 02 13:42:36.931004 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.930909 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kvknk"] Jul 02 13:42:36.959751 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.959716 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:36.959904 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.959764 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9r4l\" (UniqueName: \"kubernetes.io/projected/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-kube-api-access-s9r4l\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:36.959904 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.959792 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-data-volume\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:36.959904 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.959874 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-crio-socket\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:36.959904 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:36.959900 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:37.061134 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:37.061096 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-crio-socket\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:37.061134 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:37.061148 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:37.061383 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:37.061238 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:37.061383 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:37.061244 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-crio-socket\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:37.061383 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:37.061284 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9r4l\" (UniqueName: \"kubernetes.io/projected/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-kube-api-access-s9r4l\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:37.061383 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:37.061311 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-data-volume\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:37.061383 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:37.061349 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 13:42:37.061383 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:37.061370 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 13:42:37.061677 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:37.061437 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-kube-rbac-proxy-cm podName:0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1 nodeName:}" failed. No retries permitted until 2026-07-02 13:42:37.561405754 +0000 UTC m=+63.687542254 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kvknk" (UID: "0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1") : configmap "kube-rbac-proxy" not found Jul 02 13:42:37.061677 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:37.061456 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-insights-runtime-extractor-tls podName:0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1 nodeName:}" failed. No retries permitted until 2026-07-02 13:42:37.561447155 +0000 UTC m=+63.687583640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kvknk" (UID: "0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1") : secret "insights-runtime-extractor-tls" not found Jul 02 13:42:37.061799 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:37.061759 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-data-volume\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:37.076636 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:37.076593 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9r4l\" (UniqueName: \"kubernetes.io/projected/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-kube-api-access-s9r4l\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:37.512454 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:37.512411 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dwvk4_db8817af-4159-4667-b285-8e6325a345c4/dns/0.log" Jul 02 13:42:37.566609 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:37.566523 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:37.566609 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:37.566585 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:37.566822 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:37.566669 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 13:42:37.566822 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:37.566735 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 13:42:37.566822 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:37.566745 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-kube-rbac-proxy-cm podName:0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1 nodeName:}" failed. No retries permitted until 2026-07-02 13:42:38.566725946 +0000 UTC m=+64.692862444 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kvknk" (UID: "0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1") : configmap "kube-rbac-proxy" not found Jul 02 13:42:37.566822 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:37.566786 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-insights-runtime-extractor-tls podName:0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1 nodeName:}" failed. No retries permitted until 2026-07-02 13:42:38.566771228 +0000 UTC m=+64.692907730 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kvknk" (UID: "0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1") : secret "insights-runtime-extractor-tls" not found Jul 02 13:42:37.638988 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:37.638925 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-sz9zz" event={"ID":"85818307-0372-452d-b408-3b21c8fafb90","Type":"ContainerStarted","Data":"d9cece6055152d96a54f8d6f38b79090e4e182b5c39b0fc5ca9189977e5ecfcc"} Jul 02 13:42:37.639206 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:37.639055 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-sz9zz" event={"ID":"85818307-0372-452d-b408-3b21c8fafb90","Type":"ContainerStarted","Data":"6cb803ecc39f2cdabe506567ee6202014234f57bf1621ef36666cb92043e10cf"} Jul 02 13:42:37.656647 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:37.656589 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7b7d896497-sz9zz" podStartSLOduration=1.398870399 podStartE2EDuration="2.65656696s" totalImagesPullingTime="1.257696561s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:42:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:42:37.655604728 +0000 UTC m=+63.781741236" watchObservedRunningTime="2026-07-02 13:42:37.65656696 +0000 UTC m=+63.782703494" Jul 02 13:42:37.685315 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:37.685287 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dwvk4_db8817af-4159-4667-b285-8e6325a345c4/kube-rbac-proxy/0.log" Jul 02 13:42:38.572536 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:38.572498 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:38.573018 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:38.572623 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 13:42:38.573018 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:38.572670 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:38.573018 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:38.572698 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-kube-rbac-proxy-cm podName:0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1 nodeName:}" failed. No retries permitted until 2026-07-02 13:42:40.572678146 +0000 UTC m=+66.698814651 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kvknk" (UID: "0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1") : configmap "kube-rbac-proxy" not found Jul 02 13:42:38.573018 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:38.572781 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 13:42:38.573018 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:38.572839 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-insights-runtime-extractor-tls podName:0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1 nodeName:}" failed. No retries permitted until 2026-07-02 13:42:40.572822742 +0000 UTC m=+66.698959230 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kvknk" (UID: "0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1") : secret "insights-runtime-extractor-tls" not found Jul 02 13:42:39.086179 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.086147 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kcdlm_4d160c6f-4496-4913-a38a-26f61cd3effe/dns-node-resolver/0.log" Jul 02 13:42:39.888333 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.888310 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cpjcf_1aec8ab3-45e7-422c-985b-0f5541ed6149/node-ca/0.log" Jul 02 13:42:39.921903 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.921869 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq"] Jul 02 13:42:39.923901 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.923884 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq" Jul 02 13:42:39.926210 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.926181 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 02 13:42:39.926761 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.926740 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-bdvtf\"" Jul 02 13:42:39.926953 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.926933 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 02 13:42:39.933605 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.933582 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-8mr8c"] Jul 02 13:42:39.935624 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.935608 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" Jul 02 13:42:39.938670 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.938310 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 02 13:42:39.938670 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.938367 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 02 13:42:39.938847 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.938693 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 02 13:42:39.938904 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.938889 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-rxtcf\"" Jul 02 13:42:39.939656 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.939634 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq"] Jul 02 13:42:39.949221 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.949197 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tp7m9"] Jul 02 13:42:39.951521 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.951500 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:39.953195 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.953169 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-8mr8c"] Jul 02 13:42:39.954453 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.954433 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 02 13:42:39.954574 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.954497 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 02 13:42:39.954857 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.954832 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-g7vx5\"" Jul 02 13:42:39.955128 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.955112 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 02 13:42:39.985554 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.985526 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/15039e2a-095d-4e7a-96aa-45ab21a5b491-node-exporter-textfile\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:39.985670 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.985570 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15039e2a-095d-4e7a-96aa-45ab21a5b491-sys\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:39.985670 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.985596 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d282969-fbb0-425e-a612-9a5801ed1b30-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-h6zdq\" (UID: \"4d282969-fbb0-425e-a612-9a5801ed1b30\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq" Jul 02 13:42:39.985670 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.985620 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msp4j\" (UniqueName: \"kubernetes.io/projected/15039e2a-095d-4e7a-96aa-45ab21a5b491-kube-api-access-msp4j\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:39.985670 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.985663 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2zn8\" (UniqueName: \"kubernetes.io/projected/62c2d08c-f0b1-4860-910f-48aab8a5101f-kube-api-access-s2zn8\") pod \"kube-state-metrics-845b954455-8mr8c\" (UID: \"62c2d08c-f0b1-4860-910f-48aab8a5101f\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" Jul 02 13:42:39.985889 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.985688 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/15039e2a-095d-4e7a-96aa-45ab21a5b491-node-exporter-tls\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:39.985889 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.985737 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15039e2a-095d-4e7a-96aa-45ab21a5b491-metrics-client-ca\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:39.985889 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.985788 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62c2d08c-f0b1-4860-910f-48aab8a5101f-metrics-client-ca\") pod \"kube-state-metrics-845b954455-8mr8c\" (UID: \"62c2d08c-f0b1-4860-910f-48aab8a5101f\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" Jul 02 13:42:39.985889 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.985842 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/15039e2a-095d-4e7a-96aa-45ab21a5b491-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:39.985889 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.985868 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/15039e2a-095d-4e7a-96aa-45ab21a5b491-node-exporter-accelerators-collector-config\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:39.986158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.985895 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/62c2d08c-f0b1-4860-910f-48aab8a5101f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-8mr8c\" (UID: \"62c2d08c-f0b1-4860-910f-48aab8a5101f\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" Jul 02 13:42:39.986158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.985923 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d282969-fbb0-425e-a612-9a5801ed1b30-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-h6zdq\" (UID: \"4d282969-fbb0-425e-a612-9a5801ed1b30\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq" Jul 02 13:42:39.986158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.985952 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d282969-fbb0-425e-a612-9a5801ed1b30-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-h6zdq\" (UID: \"4d282969-fbb0-425e-a612-9a5801ed1b30\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq" Jul 02 13:42:39.986158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.986038 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/15039e2a-095d-4e7a-96aa-45ab21a5b491-root\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:39.986158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.986064 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znchs\" (UniqueName: \"kubernetes.io/projected/4d282969-fbb0-425e-a612-9a5801ed1b30-kube-api-access-znchs\") pod \"openshift-state-metrics-57f99c8b49-h6zdq\" (UID: \"4d282969-fbb0-425e-a612-9a5801ed1b30\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq" Jul 02 13:42:39.986158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.986095 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62c2d08c-f0b1-4860-910f-48aab8a5101f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-8mr8c\" (UID: \"62c2d08c-f0b1-4860-910f-48aab8a5101f\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" Jul 02 13:42:39.986450 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.986164 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/62c2d08c-f0b1-4860-910f-48aab8a5101f-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-8mr8c\" (UID: \"62c2d08c-f0b1-4860-910f-48aab8a5101f\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" Jul 02 13:42:39.986450 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.986207 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/15039e2a-095d-4e7a-96aa-45ab21a5b491-node-exporter-wtmp\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:39.986450 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:39.986231 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/62c2d08c-f0b1-4860-910f-48aab8a5101f-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-8mr8c\" (UID: \"62c2d08c-f0b1-4860-910f-48aab8a5101f\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" Jul 02 13:42:40.087097 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087060 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/62c2d08c-f0b1-4860-910f-48aab8a5101f-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-8mr8c\" (UID: \"62c2d08c-f0b1-4860-910f-48aab8a5101f\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" Jul 02 13:42:40.087097 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087102 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret\") pod \"global-pull-secret-syncer-flv4b\" (UID: \"cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f\") " pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:42:40.087340 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087126 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/15039e2a-095d-4e7a-96aa-45ab21a5b491-node-exporter-wtmp\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:40.087340 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087159 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/62c2d08c-f0b1-4860-910f-48aab8a5101f-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-8mr8c\" (UID: \"62c2d08c-f0b1-4860-910f-48aab8a5101f\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" Jul 02 13:42:40.087340 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087194 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/15039e2a-095d-4e7a-96aa-45ab21a5b491-node-exporter-textfile\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:40.087340 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087217 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15039e2a-095d-4e7a-96aa-45ab21a5b491-sys\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:40.087340 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:40.087242 2561 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 02 13:42:40.087340 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:40.087313 2561 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 02 13:42:40.087340 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:40.087321 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62c2d08c-f0b1-4860-910f-48aab8a5101f-kube-state-metrics-tls podName:62c2d08c-f0b1-4860-910f-48aab8a5101f nodeName:}" failed. No retries permitted until 2026-07-02 13:42:40.587299271 +0000 UTC m=+66.713435760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/62c2d08c-f0b1-4860-910f-48aab8a5101f-kube-state-metrics-tls") pod "kube-state-metrics-845b954455-8mr8c" (UID: "62c2d08c-f0b1-4860-910f-48aab8a5101f") : secret "kube-state-metrics-tls" not found Jul 02 13:42:40.087774 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:40.087361 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4d282969-fbb0-425e-a612-9a5801ed1b30-openshift-state-metrics-tls podName:4d282969-fbb0-425e-a612-9a5801ed1b30 nodeName:}" failed. No retries permitted until 2026-07-02 13:42:40.587345281 +0000 UTC m=+66.713481765 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/4d282969-fbb0-425e-a612-9a5801ed1b30-openshift-state-metrics-tls") pod "openshift-state-metrics-57f99c8b49-h6zdq" (UID: "4d282969-fbb0-425e-a612-9a5801ed1b30") : secret "openshift-state-metrics-tls" not found Jul 02 13:42:40.087774 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087371 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/15039e2a-095d-4e7a-96aa-45ab21a5b491-node-exporter-wtmp\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:40.087774 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087242 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d282969-fbb0-425e-a612-9a5801ed1b30-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-h6zdq\" (UID: \"4d282969-fbb0-425e-a612-9a5801ed1b30\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq" Jul 02 13:42:40.087774 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087429 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msp4j\" (UniqueName: \"kubernetes.io/projected/15039e2a-095d-4e7a-96aa-45ab21a5b491-kube-api-access-msp4j\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:40.087774 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087473 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2zn8\" (UniqueName: \"kubernetes.io/projected/62c2d08c-f0b1-4860-910f-48aab8a5101f-kube-api-access-s2zn8\") pod \"kube-state-metrics-845b954455-8mr8c\" (UID: \"62c2d08c-f0b1-4860-910f-48aab8a5101f\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" Jul 02 13:42:40.087774 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087498 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/15039e2a-095d-4e7a-96aa-45ab21a5b491-node-exporter-tls\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:40.087774 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087505 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15039e2a-095d-4e7a-96aa-45ab21a5b491-sys\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:40.087774 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087520 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15039e2a-095d-4e7a-96aa-45ab21a5b491-metrics-client-ca\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:40.087774 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087582 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62c2d08c-f0b1-4860-910f-48aab8a5101f-metrics-client-ca\") pod \"kube-state-metrics-845b954455-8mr8c\" (UID: \"62c2d08c-f0b1-4860-910f-48aab8a5101f\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" Jul 02 13:42:40.087774 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087694 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/15039e2a-095d-4e7a-96aa-45ab21a5b491-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:40.087774 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087597 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/62c2d08c-f0b1-4860-910f-48aab8a5101f-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-8mr8c\" (UID: \"62c2d08c-f0b1-4860-910f-48aab8a5101f\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" Jul 02 13:42:40.087774 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087610 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/15039e2a-095d-4e7a-96aa-45ab21a5b491-node-exporter-textfile\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:40.087774 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087728 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/15039e2a-095d-4e7a-96aa-45ab21a5b491-node-exporter-accelerators-collector-config\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:40.087774 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087779 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/62c2d08c-f0b1-4860-910f-48aab8a5101f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-8mr8c\" (UID: \"62c2d08c-f0b1-4860-910f-48aab8a5101f\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" Jul 02 13:42:40.088468 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087809 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d282969-fbb0-425e-a612-9a5801ed1b30-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-h6zdq\" (UID: \"4d282969-fbb0-425e-a612-9a5801ed1b30\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq" Jul 02 13:42:40.088468 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087840 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d282969-fbb0-425e-a612-9a5801ed1b30-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-h6zdq\" (UID: \"4d282969-fbb0-425e-a612-9a5801ed1b30\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq" Jul 02 13:42:40.088468 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087881 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/15039e2a-095d-4e7a-96aa-45ab21a5b491-root\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:40.088468 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087928 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znchs\" (UniqueName: \"kubernetes.io/projected/4d282969-fbb0-425e-a612-9a5801ed1b30-kube-api-access-znchs\") pod \"openshift-state-metrics-57f99c8b49-h6zdq\" (UID: \"4d282969-fbb0-425e-a612-9a5801ed1b30\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq" Jul 02 13:42:40.088468 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.087961 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62c2d08c-f0b1-4860-910f-48aab8a5101f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-8mr8c\" (UID: \"62c2d08c-f0b1-4860-910f-48aab8a5101f\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" Jul 02 13:42:40.088468 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.088147 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/15039e2a-095d-4e7a-96aa-45ab21a5b491-metrics-client-ca\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:40.088468 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:40.087613 2561 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 02 13:42:40.088468 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:40.088279 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15039e2a-095d-4e7a-96aa-45ab21a5b491-node-exporter-tls podName:15039e2a-095d-4e7a-96aa-45ab21a5b491 nodeName:}" failed. No retries permitted until 2026-07-02 13:42:40.588255041 +0000 UTC m=+66.714391524 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/15039e2a-095d-4e7a-96aa-45ab21a5b491-node-exporter-tls") pod "node-exporter-tp7m9" (UID: "15039e2a-095d-4e7a-96aa-45ab21a5b491") : secret "node-exporter-tls" not found Jul 02 13:42:40.088468 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.088333 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/15039e2a-095d-4e7a-96aa-45ab21a5b491-node-exporter-accelerators-collector-config\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:40.088468 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.088365 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62c2d08c-f0b1-4860-910f-48aab8a5101f-metrics-client-ca\") pod \"kube-state-metrics-845b954455-8mr8c\" (UID: \"62c2d08c-f0b1-4860-910f-48aab8a5101f\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" Jul 02 13:42:40.088468 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.088393 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/15039e2a-095d-4e7a-96aa-45ab21a5b491-root\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:40.089018 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.088843 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/62c2d08c-f0b1-4860-910f-48aab8a5101f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-8mr8c\" (UID: \"62c2d08c-f0b1-4860-910f-48aab8a5101f\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" Jul 02 13:42:40.089018 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.088915 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d282969-fbb0-425e-a612-9a5801ed1b30-metrics-client-ca\") pod \"openshift-state-metrics-57f99c8b49-h6zdq\" (UID: \"4d282969-fbb0-425e-a612-9a5801ed1b30\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq" Jul 02 13:42:40.089652 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.089635 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 13:42:40.090582 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.090558 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/15039e2a-095d-4e7a-96aa-45ab21a5b491-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:40.090887 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.090868 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62c2d08c-f0b1-4860-910f-48aab8a5101f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-8mr8c\" (UID: \"62c2d08c-f0b1-4860-910f-48aab8a5101f\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" Jul 02 13:42:40.090990 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.090954 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4d282969-fbb0-425e-a612-9a5801ed1b30-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-57f99c8b49-h6zdq\" (UID: \"4d282969-fbb0-425e-a612-9a5801ed1b30\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq" Jul 02 13:42:40.098643 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.098618 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2zn8\" (UniqueName: \"kubernetes.io/projected/62c2d08c-f0b1-4860-910f-48aab8a5101f-kube-api-access-s2zn8\") pod \"kube-state-metrics-845b954455-8mr8c\" (UID: \"62c2d08c-f0b1-4860-910f-48aab8a5101f\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" Jul 02 13:42:40.099202 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.099148 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msp4j\" (UniqueName: \"kubernetes.io/projected/15039e2a-095d-4e7a-96aa-45ab21a5b491-kube-api-access-msp4j\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:40.099714 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.099691 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-znchs\" (UniqueName: \"kubernetes.io/projected/4d282969-fbb0-425e-a612-9a5801ed1b30-kube-api-access-znchs\") pod \"openshift-state-metrics-57f99c8b49-h6zdq\" (UID: \"4d282969-fbb0-425e-a612-9a5801ed1b30\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq" Jul 02 13:42:40.100341 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.100318 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f-original-pull-secret\") pod \"global-pull-secret-syncer-flv4b\" (UID: \"cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f\") " pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:42:40.174128 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.174029 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-flv4b" Jul 02 13:42:40.188770 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.188737 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs\") pod \"network-metrics-daemon-j9rh2\" (UID: \"da0298d6-5c20-459a-864b-0a3146e256ee\") " pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:42:40.191238 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.191216 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 13:42:40.202478 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.202392 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/da0298d6-5c20-459a-864b-0a3146e256ee-metrics-certs\") pod \"network-metrics-daemon-j9rh2\" (UID: \"da0298d6-5c20-459a-864b-0a3146e256ee\") " pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:42:40.289654 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.289563 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gsz6\" (UniqueName: \"kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6\") pod \"network-check-target-jv42p\" (UID: \"684ebf83-fb83-47a1-94c7-ecd7bba444c7\") " pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:42:40.292080 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.291956 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 13:42:40.302447 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.302424 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 13:42:40.313348 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.313323 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gsz6\" (UniqueName: \"kubernetes.io/projected/684ebf83-fb83-47a1-94c7-ecd7bba444c7-kube-api-access-9gsz6\") pod \"network-check-target-jv42p\" (UID: \"684ebf83-fb83-47a1-94c7-ecd7bba444c7\") " pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:42:40.322841 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.322817 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-flv4b"] Jul 02 13:42:40.325601 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:42:40.325574 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf9efe36_5e20_4b2a_bda2_5eb6117bcb3f.slice/crio-5c17dc85842f4a26e1892417b71d9182d980cbe27dd55aa48b9f97edaffc59f4 WatchSource:0}: Error finding container 5c17dc85842f4a26e1892417b71d9182d980cbe27dd55aa48b9f97edaffc59f4: Status 404 returned error can't find the container with id 5c17dc85842f4a26e1892417b71d9182d980cbe27dd55aa48b9f97edaffc59f4 Jul 02 13:42:40.470096 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.470064 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bhzn5\"" Jul 02 13:42:40.478255 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.478209 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-j9rh2" Jul 02 13:42:40.481156 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.480964 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-tz79z\"" Jul 02 13:42:40.489088 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.489040 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:42:40.602000 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.594249 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/62c2d08c-f0b1-4860-910f-48aab8a5101f-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-8mr8c\" (UID: \"62c2d08c-f0b1-4860-910f-48aab8a5101f\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" Jul 02 13:42:40.602000 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.594329 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d282969-fbb0-425e-a612-9a5801ed1b30-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-h6zdq\" (UID: \"4d282969-fbb0-425e-a612-9a5801ed1b30\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq" Jul 02 13:42:40.602000 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.594365 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:40.602000 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.594393 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/15039e2a-095d-4e7a-96aa-45ab21a5b491-node-exporter-tls\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:40.602000 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.594440 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:40.602000 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:40.594558 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 13:42:40.602000 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:40.594621 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-kube-rbac-proxy-cm podName:0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1 nodeName:}" failed. No retries permitted until 2026-07-02 13:42:44.594601068 +0000 UTC m=+70.720737567 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-kvknk" (UID: "0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1") : configmap "kube-rbac-proxy" not found Jul 02 13:42:40.602000 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:40.601186 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 13:42:40.602000 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:40.601266 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-insights-runtime-extractor-tls podName:0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1 nodeName:}" failed. No retries permitted until 2026-07-02 13:42:44.601244176 +0000 UTC m=+70.727380674 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-kvknk" (UID: "0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1") : secret "insights-runtime-extractor-tls" not found Jul 02 13:42:40.605502 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.605442 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/15039e2a-095d-4e7a-96aa-45ab21a5b491-node-exporter-tls\") pod \"node-exporter-tp7m9\" (UID: \"15039e2a-095d-4e7a-96aa-45ab21a5b491\") " pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:40.620457 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.618061 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/62c2d08c-f0b1-4860-910f-48aab8a5101f-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-8mr8c\" (UID: \"62c2d08c-f0b1-4860-910f-48aab8a5101f\") " pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" Jul 02 13:42:40.620457 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.620384 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4d282969-fbb0-425e-a612-9a5801ed1b30-openshift-state-metrics-tls\") pod \"openshift-state-metrics-57f99c8b49-h6zdq\" (UID: \"4d282969-fbb0-425e-a612-9a5801ed1b30\") " pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq" Jul 02 13:42:40.644217 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.644186 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-j9rh2"] Jul 02 13:42:40.649697 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:42:40.649663 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda0298d6_5c20_459a_864b_0a3146e256ee.slice/crio-cc487adbb472125c1663f1fb7ecb0a9e11943ededa82863b9372e16fce0cce1c WatchSource:0}: Error finding container cc487adbb472125c1663f1fb7ecb0a9e11943ededa82863b9372e16fce0cce1c: Status 404 returned error can't find the container with id cc487adbb472125c1663f1fb7ecb0a9e11943ededa82863b9372e16fce0cce1c Jul 02 13:42:40.651500 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.651377 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-flv4b" event={"ID":"cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f","Type":"ContainerStarted","Data":"5c17dc85842f4a26e1892417b71d9182d980cbe27dd55aa48b9f97edaffc59f4"} Jul 02 13:42:40.674294 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.674258 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jv42p"] Jul 02 13:42:40.679373 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:42:40.678830 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod684ebf83_fb83_47a1_94c7_ecd7bba444c7.slice/crio-375d6978c0daaffe5f806d933c8313e3f81411543693096b78b5ac0f212282e4 WatchSource:0}: Error finding container 375d6978c0daaffe5f806d933c8313e3f81411543693096b78b5ac0f212282e4: Status 404 returned error can't find the container with id 375d6978c0daaffe5f806d933c8313e3f81411543693096b78b5ac0f212282e4 Jul 02 13:42:40.833014 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.832924 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq" Jul 02 13:42:40.846720 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.846689 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" Jul 02 13:42:40.863069 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.863027 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tp7m9" Jul 02 13:42:40.877222 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:42:40.877192 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15039e2a_095d_4e7a_96aa_45ab21a5b491.slice/crio-2ebb0818ea3325a2cdf7ab7242c36ff3aa2d6e58bd61ff70a19de815f77590df WatchSource:0}: Error finding container 2ebb0818ea3325a2cdf7ab7242c36ff3aa2d6e58bd61ff70a19de815f77590df: Status 404 returned error can't find the container with id 2ebb0818ea3325a2cdf7ab7242c36ff3aa2d6e58bd61ff70a19de815f77590df Jul 02 13:42:40.884729 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:40.884675 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dwb5c_a509cdb7-1c6e-4e40-9add-d802b749c4b0/serve-healthcheck-canary/0.log" Jul 02 13:42:41.044057 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.043907 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-8mr8c"] Jul 02 13:42:41.046132 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.045681 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq"] Jul 02 13:42:41.049255 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:42:41.048948 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d282969_fbb0_425e_a612_9a5801ed1b30.slice/crio-e93498f9c508c267b952af285e4c4c2e5754cf4d1db8cbd8c12c14680b86daca WatchSource:0}: Error finding container e93498f9c508c267b952af285e4c4c2e5754cf4d1db8cbd8c12c14680b86daca: Status 404 returned error can't find the container with id e93498f9c508c267b952af285e4c4c2e5754cf4d1db8cbd8c12c14680b86daca Jul 02 13:42:41.049829 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:42:41.049789 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62c2d08c_f0b1_4860_910f_48aab8a5101f.slice/crio-8612b37fc53059e9c4cc53ff6baab4e3c63f4ce940be2cd1bb80ef2c09bb0f68 WatchSource:0}: Error finding container 8612b37fc53059e9c4cc53ff6baab4e3c63f4ce940be2cd1bb80ef2c09bb0f68: Status 404 returned error can't find the container with id 8612b37fc53059e9c4cc53ff6baab4e3c63f4ce940be2cd1bb80ef2c09bb0f68 Jul 02 13:42:41.080835 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.080746 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 13:42:41.099464 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.096642 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.103003 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.101739 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 02 13:42:41.103336 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.103221 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 02 13:42:41.103336 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.103273 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 02 13:42:41.103597 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.103487 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 02 13:42:41.104243 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.103757 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 02 13:42:41.104243 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.104026 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 02 13:42:41.104243 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.104225 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-hmb8n\"" Jul 02 13:42:41.105001 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.104467 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 02 13:42:41.105001 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.104907 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 02 13:42:41.106038 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.105191 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 02 13:42:41.106038 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.105930 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 13:42:41.200696 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.200663 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.201006 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.200939 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef56f414-16ad-4597-83e3-46b3f1e93c15-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.201214 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.201197 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.201371 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.201356 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef56f414-16ad-4597-83e3-46b3f1e93c15-config-out\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.201517 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.201502 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef56f414-16ad-4597-83e3-46b3f1e93c15-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.201677 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.201655 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-config-volume\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.201846 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.201798 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.202016 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.201960 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-web-config\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.202214 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.202196 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ef56f414-16ad-4597-83e3-46b3f1e93c15-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.202381 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.202365 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.202500 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.202486 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef56f414-16ad-4597-83e3-46b3f1e93c15-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.202630 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.202616 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.202776 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.202761 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwjtb\" (UniqueName: \"kubernetes.io/projected/ef56f414-16ad-4597-83e3-46b3f1e93c15-kube-api-access-wwjtb\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.303685 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.303643 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.303854 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.303702 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef56f414-16ad-4597-83e3-46b3f1e93c15-config-out\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.303854 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.303729 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef56f414-16ad-4597-83e3-46b3f1e93c15-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.303854 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.303750 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-config-volume\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.303854 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.303779 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.303854 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.303811 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-web-config\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.303854 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.303837 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ef56f414-16ad-4597-83e3-46b3f1e93c15-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.304229 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.303883 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.304229 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.303912 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef56f414-16ad-4597-83e3-46b3f1e93c15-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.304229 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.303947 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.304229 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.303994 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwjtb\" (UniqueName: \"kubernetes.io/projected/ef56f414-16ad-4597-83e3-46b3f1e93c15-kube-api-access-wwjtb\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.304229 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.304033 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.304229 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.304069 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef56f414-16ad-4597-83e3-46b3f1e93c15-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.304514 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:41.304421 2561 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 02 13:42:41.304514 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:42:41.304495 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-main-tls podName:ef56f414-16ad-4597-83e3-46b3f1e93c15 nodeName:}" failed. No retries permitted until 2026-07-02 13:42:41.804475244 +0000 UTC m=+67.930611729 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "ef56f414-16ad-4597-83e3-46b3f1e93c15") : secret "alertmanager-main-tls" not found Jul 02 13:42:41.305027 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.305003 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ef56f414-16ad-4597-83e3-46b3f1e93c15-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.305790 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.305222 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef56f414-16ad-4597-83e3-46b3f1e93c15-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.305790 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.305297 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef56f414-16ad-4597-83e3-46b3f1e93c15-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.310550 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.310430 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-web-config\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.311219 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.311133 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-config-volume\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.313602 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.313552 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef56f414-16ad-4597-83e3-46b3f1e93c15-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.314677 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.314634 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.314789 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.314743 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.314850 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.314797 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.315347 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.315322 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.317083 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.316994 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwjtb\" (UniqueName: \"kubernetes.io/projected/ef56f414-16ad-4597-83e3-46b3f1e93c15-kube-api-access-wwjtb\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.320432 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.320381 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef56f414-16ad-4597-83e3-46b3f1e93c15-config-out\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.659831 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.659677 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tp7m9" event={"ID":"15039e2a-095d-4e7a-96aa-45ab21a5b491","Type":"ContainerStarted","Data":"2ebb0818ea3325a2cdf7ab7242c36ff3aa2d6e58bd61ff70a19de815f77590df"} Jul 02 13:42:41.661249 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.661214 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" event={"ID":"62c2d08c-f0b1-4860-910f-48aab8a5101f","Type":"ContainerStarted","Data":"8612b37fc53059e9c4cc53ff6baab4e3c63f4ce940be2cd1bb80ef2c09bb0f68"} Jul 02 13:42:41.664362 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.664334 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq" event={"ID":"4d282969-fbb0-425e-a612-9a5801ed1b30","Type":"ContainerStarted","Data":"f09c6144036e7fbf0e33498a8f4fcb06eb6b94e49dbff8bce1a5c007379ba1fe"} Jul 02 13:42:41.664461 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.664394 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq" event={"ID":"4d282969-fbb0-425e-a612-9a5801ed1b30","Type":"ContainerStarted","Data":"b0498437762646608e498143493cf22d4b6a95f1a3230724d6cafa7c5942722f"} Jul 02 13:42:41.664461 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.664423 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq" event={"ID":"4d282969-fbb0-425e-a612-9a5801ed1b30","Type":"ContainerStarted","Data":"e93498f9c508c267b952af285e4c4c2e5754cf4d1db8cbd8c12c14680b86daca"} Jul 02 13:42:41.666512 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.666486 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jv42p" event={"ID":"684ebf83-fb83-47a1-94c7-ecd7bba444c7","Type":"ContainerStarted","Data":"375d6978c0daaffe5f806d933c8313e3f81411543693096b78b5ac0f212282e4"} Jul 02 13:42:41.668552 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.668515 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j9rh2" event={"ID":"da0298d6-5c20-459a-864b-0a3146e256ee","Type":"ContainerStarted","Data":"cc487adbb472125c1663f1fb7ecb0a9e11943ededa82863b9372e16fce0cce1c"} Jul 02 13:42:41.808640 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.808603 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:41.811670 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:41.811646 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:42.047025 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:42.046987 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:42:42.623433 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:42.623401 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dwvk4" Jul 02 13:42:42.995847 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:42.995815 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7"] Jul 02 13:42:42.998929 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:42.998904 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.003896 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.003867 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 02 13:42:43.004154 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.004137 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 02 13:42:43.004476 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.004460 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-5fsx7\"" Jul 02 13:42:43.004630 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.004613 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 02 13:42:43.004729 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.004704 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-8kbj4q1l05pbb\"" Jul 02 13:42:43.004838 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.004822 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 02 13:42:43.005069 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.005041 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 02 13:42:43.014753 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.014418 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7"] Jul 02 13:42:43.021614 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.021585 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnwtx\" (UniqueName: \"kubernetes.io/projected/a783602a-9039-4d41-ad1f-13cedff176cf-kube-api-access-dnwtx\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.021750 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.021631 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a783602a-9039-4d41-ad1f-13cedff176cf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.021750 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.021694 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a783602a-9039-4d41-ad1f-13cedff176cf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.021750 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.021736 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a783602a-9039-4d41-ad1f-13cedff176cf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.021921 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.021789 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a783602a-9039-4d41-ad1f-13cedff176cf-secret-thanos-querier-tls\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.021921 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.021811 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a783602a-9039-4d41-ad1f-13cedff176cf-secret-grpc-tls\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.021921 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.021849 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a783602a-9039-4d41-ad1f-13cedff176cf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.021921 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.021877 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a783602a-9039-4d41-ad1f-13cedff176cf-metrics-client-ca\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.123408 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.123365 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a783602a-9039-4d41-ad1f-13cedff176cf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.123875 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.123434 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a783602a-9039-4d41-ad1f-13cedff176cf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.123875 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.123487 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a783602a-9039-4d41-ad1f-13cedff176cf-secret-thanos-querier-tls\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.123875 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.123513 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a783602a-9039-4d41-ad1f-13cedff176cf-secret-grpc-tls\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.123875 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.123555 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a783602a-9039-4d41-ad1f-13cedff176cf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.123875 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.123579 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a783602a-9039-4d41-ad1f-13cedff176cf-metrics-client-ca\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.123875 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.123637 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnwtx\" (UniqueName: \"kubernetes.io/projected/a783602a-9039-4d41-ad1f-13cedff176cf-kube-api-access-dnwtx\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.123875 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.123669 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a783602a-9039-4d41-ad1f-13cedff176cf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.125432 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.125399 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a783602a-9039-4d41-ad1f-13cedff176cf-metrics-client-ca\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.129431 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.129396 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a783602a-9039-4d41-ad1f-13cedff176cf-secret-grpc-tls\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.129577 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.129517 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a783602a-9039-4d41-ad1f-13cedff176cf-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.130731 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.130274 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/a783602a-9039-4d41-ad1f-13cedff176cf-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.130731 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.130693 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a783602a-9039-4d41-ad1f-13cedff176cf-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.131517 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.131493 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/a783602a-9039-4d41-ad1f-13cedff176cf-secret-thanos-querier-tls\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.134407 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.134353 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/a783602a-9039-4d41-ad1f-13cedff176cf-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.137449 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.137397 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnwtx\" (UniqueName: \"kubernetes.io/projected/a783602a-9039-4d41-ad1f-13cedff176cf-kube-api-access-dnwtx\") pod \"thanos-querier-5d4bbc8bd-bb7t7\" (UID: \"a783602a-9039-4d41-ad1f-13cedff176cf\") " pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.202779 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.201435 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5896977f4b-hmj9w"] Jul 02 13:42:43.204557 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.204530 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:42:43.209783 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.209412 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 02 13:42:43.211114 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.210172 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 02 13:42:43.211114 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.210391 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 02 13:42:43.211114 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.210609 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 02 13:42:43.211114 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.210804 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-56nzm\"" Jul 02 13:42:43.211114 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.211048 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 02 13:42:43.220881 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.220838 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5896977f4b-hmj9w"] Jul 02 13:42:43.224730 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.224683 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea824dc9-396f-445a-a895-d255dd7e5641-service-ca\") pod \"console-5896977f4b-hmj9w\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:42:43.224891 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.224745 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea824dc9-396f-445a-a895-d255dd7e5641-oauth-serving-cert\") pod \"console-5896977f4b-hmj9w\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:42:43.224891 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.224807 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea824dc9-396f-445a-a895-d255dd7e5641-console-config\") pod \"console-5896977f4b-hmj9w\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:42:43.224891 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.224867 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26f8c\" (UniqueName: \"kubernetes.io/projected/ea824dc9-396f-445a-a895-d255dd7e5641-kube-api-access-26f8c\") pod \"console-5896977f4b-hmj9w\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:42:43.225098 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.224915 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea824dc9-396f-445a-a895-d255dd7e5641-console-oauth-config\") pod \"console-5896977f4b-hmj9w\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:42:43.225098 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.225011 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea824dc9-396f-445a-a895-d255dd7e5641-console-serving-cert\") pod \"console-5896977f4b-hmj9w\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:42:43.322959 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.322209 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:42:43.326064 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.325659 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea824dc9-396f-445a-a895-d255dd7e5641-console-config\") pod \"console-5896977f4b-hmj9w\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:42:43.326064 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.325710 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26f8c\" (UniqueName: \"kubernetes.io/projected/ea824dc9-396f-445a-a895-d255dd7e5641-kube-api-access-26f8c\") pod \"console-5896977f4b-hmj9w\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:42:43.326064 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.325742 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea824dc9-396f-445a-a895-d255dd7e5641-console-oauth-config\") pod \"console-5896977f4b-hmj9w\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:42:43.326064 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.325799 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea824dc9-396f-445a-a895-d255dd7e5641-console-serving-cert\") pod \"console-5896977f4b-hmj9w\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:42:43.326064 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.325868 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea824dc9-396f-445a-a895-d255dd7e5641-service-ca\") pod \"console-5896977f4b-hmj9w\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:42:43.326064 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.325893 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea824dc9-396f-445a-a895-d255dd7e5641-oauth-serving-cert\") pod \"console-5896977f4b-hmj9w\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:42:43.326538 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.326466 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea824dc9-396f-445a-a895-d255dd7e5641-console-config\") pod \"console-5896977f4b-hmj9w\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:42:43.327233 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.327190 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea824dc9-396f-445a-a895-d255dd7e5641-oauth-serving-cert\") pod \"console-5896977f4b-hmj9w\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:42:43.327462 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.327442 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea824dc9-396f-445a-a895-d255dd7e5641-service-ca\") pod \"console-5896977f4b-hmj9w\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:42:43.333527 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.333398 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea824dc9-396f-445a-a895-d255dd7e5641-console-serving-cert\") pod \"console-5896977f4b-hmj9w\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:42:43.334539 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.334483 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea824dc9-396f-445a-a895-d255dd7e5641-console-oauth-config\") pod \"console-5896977f4b-hmj9w\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:42:43.344281 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.344209 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26f8c\" (UniqueName: \"kubernetes.io/projected/ea824dc9-396f-445a-a895-d255dd7e5641-kube-api-access-26f8c\") pod \"console-5896977f4b-hmj9w\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:42:43.522051 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.522012 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:42:43.807906 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:43.807762 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 13:42:44.277297 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.277205 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-864ff6486b-zhmhz"] Jul 02 13:42:44.280766 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.280740 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.284312 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.284123 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-pk6xj\"" Jul 02 13:42:44.284312 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.284191 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 02 13:42:44.284312 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.284285 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 02 13:42:44.284949 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.284837 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 02 13:42:44.284949 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.284851 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tm83g731jf30\"" Jul 02 13:42:44.284949 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.284871 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 02 13:42:44.301932 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.301883 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-864ff6486b-zhmhz"] Jul 02 13:42:44.335752 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.335585 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/716d8090-1589-4a66-9784-fdd03c5e4358-secret-metrics-server-client-certs\") pod \"metrics-server-864ff6486b-zhmhz\" (UID: \"716d8090-1589-4a66-9784-fdd03c5e4358\") " pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.335752 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.335630 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/716d8090-1589-4a66-9784-fdd03c5e4358-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-864ff6486b-zhmhz\" (UID: \"716d8090-1589-4a66-9784-fdd03c5e4358\") " pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.335752 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.335671 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/716d8090-1589-4a66-9784-fdd03c5e4358-secret-metrics-server-tls\") pod \"metrics-server-864ff6486b-zhmhz\" (UID: \"716d8090-1589-4a66-9784-fdd03c5e4358\") " pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.335752 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.335688 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/716d8090-1589-4a66-9784-fdd03c5e4358-client-ca-bundle\") pod \"metrics-server-864ff6486b-zhmhz\" (UID: \"716d8090-1589-4a66-9784-fdd03c5e4358\") " pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.335752 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.335750 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/716d8090-1589-4a66-9784-fdd03c5e4358-metrics-server-audit-profiles\") pod \"metrics-server-864ff6486b-zhmhz\" (UID: \"716d8090-1589-4a66-9784-fdd03c5e4358\") " pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.336178 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.335850 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4klkz\" (UniqueName: \"kubernetes.io/projected/716d8090-1589-4a66-9784-fdd03c5e4358-kube-api-access-4klkz\") pod \"metrics-server-864ff6486b-zhmhz\" (UID: \"716d8090-1589-4a66-9784-fdd03c5e4358\") " pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.336178 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.335888 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/716d8090-1589-4a66-9784-fdd03c5e4358-audit-log\") pod \"metrics-server-864ff6486b-zhmhz\" (UID: \"716d8090-1589-4a66-9784-fdd03c5e4358\") " pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.436663 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.436624 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4klkz\" (UniqueName: \"kubernetes.io/projected/716d8090-1589-4a66-9784-fdd03c5e4358-kube-api-access-4klkz\") pod \"metrics-server-864ff6486b-zhmhz\" (UID: \"716d8090-1589-4a66-9784-fdd03c5e4358\") " pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.436663 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.436668 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/716d8090-1589-4a66-9784-fdd03c5e4358-audit-log\") pod \"metrics-server-864ff6486b-zhmhz\" (UID: \"716d8090-1589-4a66-9784-fdd03c5e4358\") " pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.436908 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.436714 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/716d8090-1589-4a66-9784-fdd03c5e4358-secret-metrics-server-client-certs\") pod \"metrics-server-864ff6486b-zhmhz\" (UID: \"716d8090-1589-4a66-9784-fdd03c5e4358\") " pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.436908 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.436780 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/716d8090-1589-4a66-9784-fdd03c5e4358-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-864ff6486b-zhmhz\" (UID: \"716d8090-1589-4a66-9784-fdd03c5e4358\") " pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.436908 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.436830 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/716d8090-1589-4a66-9784-fdd03c5e4358-secret-metrics-server-tls\") pod \"metrics-server-864ff6486b-zhmhz\" (UID: \"716d8090-1589-4a66-9784-fdd03c5e4358\") " pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.436908 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.436856 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/716d8090-1589-4a66-9784-fdd03c5e4358-client-ca-bundle\") pod \"metrics-server-864ff6486b-zhmhz\" (UID: \"716d8090-1589-4a66-9784-fdd03c5e4358\") " pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.436908 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.436890 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/716d8090-1589-4a66-9784-fdd03c5e4358-metrics-server-audit-profiles\") pod \"metrics-server-864ff6486b-zhmhz\" (UID: \"716d8090-1589-4a66-9784-fdd03c5e4358\") " pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.437153 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.437134 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/716d8090-1589-4a66-9784-fdd03c5e4358-audit-log\") pod \"metrics-server-864ff6486b-zhmhz\" (UID: \"716d8090-1589-4a66-9784-fdd03c5e4358\") " pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.437781 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.437752 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/716d8090-1589-4a66-9784-fdd03c5e4358-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-864ff6486b-zhmhz\" (UID: \"716d8090-1589-4a66-9784-fdd03c5e4358\") " pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.438017 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.437993 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/716d8090-1589-4a66-9784-fdd03c5e4358-metrics-server-audit-profiles\") pod \"metrics-server-864ff6486b-zhmhz\" (UID: \"716d8090-1589-4a66-9784-fdd03c5e4358\") " pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.439935 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.439890 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/716d8090-1589-4a66-9784-fdd03c5e4358-secret-metrics-server-client-certs\") pod \"metrics-server-864ff6486b-zhmhz\" (UID: \"716d8090-1589-4a66-9784-fdd03c5e4358\") " pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.440056 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.439956 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/716d8090-1589-4a66-9784-fdd03c5e4358-secret-metrics-server-tls\") pod \"metrics-server-864ff6486b-zhmhz\" (UID: \"716d8090-1589-4a66-9784-fdd03c5e4358\") " pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.440599 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.440559 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/716d8090-1589-4a66-9784-fdd03c5e4358-client-ca-bundle\") pod \"metrics-server-864ff6486b-zhmhz\" (UID: \"716d8090-1589-4a66-9784-fdd03c5e4358\") " pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.446275 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.446252 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4klkz\" (UniqueName: \"kubernetes.io/projected/716d8090-1589-4a66-9784-fdd03c5e4358-kube-api-access-4klkz\") pod \"metrics-server-864ff6486b-zhmhz\" (UID: \"716d8090-1589-4a66-9784-fdd03c5e4358\") " pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.593499 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.593404 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:42:44.638144 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.638106 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:44.638332 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.638236 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:44.638776 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.638747 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:44.640847 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.640826 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-kvknk\" (UID: \"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1\") " pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:44.729686 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.729652 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-sg225"] Jul 02 13:42:44.729865 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.729839 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-kvknk" Jul 02 13:42:44.733286 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.733259 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-sg225" Jul 02 13:42:44.735712 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.735663 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-ss7xs\"" Jul 02 13:42:44.735712 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.735705 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 02 13:42:44.742394 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.742369 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-sg225"] Jul 02 13:42:44.839616 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.839494 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr95z\" (UniqueName: \"kubernetes.io/projected/8e465295-97e6-498f-a211-e54e6bad7273-kube-api-access-cr95z\") pod \"monitoring-plugin-cd8656b8-sg225\" (UID: \"8e465295-97e6-498f-a211-e54e6bad7273\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-sg225" Jul 02 13:42:44.839616 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.839585 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8e465295-97e6-498f-a211-e54e6bad7273-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-sg225\" (UID: \"8e465295-97e6-498f-a211-e54e6bad7273\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-sg225" Jul 02 13:42:44.912296 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:42:44.912258 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef56f414_16ad_4597_83e3_46b3f1e93c15.slice/crio-23b4df46cb4a7b89fc6a0f73b58e22afa6d7c1f1704e0c90915e930785718a21 WatchSource:0}: Error finding container 23b4df46cb4a7b89fc6a0f73b58e22afa6d7c1f1704e0c90915e930785718a21: Status 404 returned error can't find the container with id 23b4df46cb4a7b89fc6a0f73b58e22afa6d7c1f1704e0c90915e930785718a21 Jul 02 13:42:44.941107 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.941084 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr95z\" (UniqueName: \"kubernetes.io/projected/8e465295-97e6-498f-a211-e54e6bad7273-kube-api-access-cr95z\") pod \"monitoring-plugin-cd8656b8-sg225\" (UID: \"8e465295-97e6-498f-a211-e54e6bad7273\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-sg225" Jul 02 13:42:44.941226 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.941138 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8e465295-97e6-498f-a211-e54e6bad7273-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-sg225\" (UID: \"8e465295-97e6-498f-a211-e54e6bad7273\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-sg225" Jul 02 13:42:44.944324 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.944274 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8e465295-97e6-498f-a211-e54e6bad7273-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-sg225\" (UID: \"8e465295-97e6-498f-a211-e54e6bad7273\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-sg225" Jul 02 13:42:44.951050 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:44.951025 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr95z\" (UniqueName: \"kubernetes.io/projected/8e465295-97e6-498f-a211-e54e6bad7273-kube-api-access-cr95z\") pod \"monitoring-plugin-cd8656b8-sg225\" (UID: \"8e465295-97e6-498f-a211-e54e6bad7273\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-sg225" Jul 02 13:42:45.052046 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:45.051999 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-sg225" Jul 02 13:42:45.688693 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:45.688648 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ef56f414-16ad-4597-83e3-46b3f1e93c15","Type":"ContainerStarted","Data":"23b4df46cb4a7b89fc6a0f73b58e22afa6d7c1f1704e0c90915e930785718a21"} Jul 02 13:42:46.260579 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.260395 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 13:42:46.264997 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.264943 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.270245 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.268239 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 02 13:42:46.270245 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.268471 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 02 13:42:46.270245 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.268635 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 02 13:42:46.270245 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.269144 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 02 13:42:46.270245 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.269354 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 02 13:42:46.270245 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.270059 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 02 13:42:46.270825 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.270561 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4mfd3ag069ovf\"" Jul 02 13:42:46.270825 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.270579 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2jnnl\"" Jul 02 13:42:46.270825 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.270762 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 02 13:42:46.272174 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.271244 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 02 13:42:46.272174 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.272000 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 02 13:42:46.272380 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.272266 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 02 13:42:46.272676 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.272652 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 02 13:42:46.273022 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.272946 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 02 13:42:46.273836 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.273808 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 02 13:42:46.279878 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.279850 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 02 13:42:46.283443 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.283415 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 13:42:46.354875 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.354838 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.355057 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.354923 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.355057 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.354963 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.355145 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.355061 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/834585dd-3bd9-4116-af0e-22bb1287a590-config-out\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.355145 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.355098 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.355145 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.355125 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-config\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.355277 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.355160 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.355277 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.355185 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.355277 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.355224 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.355277 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.355259 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.355489 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.355290 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.355489 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.355315 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.355489 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.355402 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/834585dd-3bd9-4116-af0e-22bb1287a590-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.355664 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.355486 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hx6z\" (UniqueName: \"kubernetes.io/projected/834585dd-3bd9-4116-af0e-22bb1287a590-kube-api-access-8hx6z\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.355664 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.355534 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.355664 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.355557 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.355664 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.355583 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.355940 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.355660 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-web-config\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.355940 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.355704 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.355940 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.355732 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.456423 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.456382 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.456423 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.456450 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.456800 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.456482 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.456800 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.456527 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/834585dd-3bd9-4116-af0e-22bb1287a590-config-out\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.456800 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.456552 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.456800 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.456577 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-config\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.456800 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.456607 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.456800 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.456633 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.456800 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.456664 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.456800 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.456706 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.456800 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.456734 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.456800 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.456763 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.456800 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.456787 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/834585dd-3bd9-4116-af0e-22bb1287a590-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.457565 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.456828 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hx6z\" (UniqueName: \"kubernetes.io/projected/834585dd-3bd9-4116-af0e-22bb1287a590-kube-api-access-8hx6z\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.457565 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.456871 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.457565 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.456887 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.457565 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.456895 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.457565 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.456954 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.457565 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.457018 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-web-config\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.457565 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.457060 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.457565 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.457086 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.457565 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.457510 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.458060 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.457874 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.458060 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.458033 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.460266 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.460240 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.460915 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.460889 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.461794 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.461757 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.462008 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.461956 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.462701 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.462620 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.463536 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.463105 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.463536 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.463500 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.465568 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.465527 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/834585dd-3bd9-4116-af0e-22bb1287a590-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.467338 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.467315 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-web-config\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.467516 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.467466 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.467615 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.467538 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.467677 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.467654 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/834585dd-3bd9-4116-af0e-22bb1287a590-config-out\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.467872 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.467826 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-config\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.468320 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.468277 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.469833 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.469787 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.470026 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.469966 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hx6z\" (UniqueName: \"kubernetes.io/projected/834585dd-3bd9-4116-af0e-22bb1287a590-kube-api-access-8hx6z\") pod \"prometheus-k8s-0\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.582444 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.582170 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:42:46.963346 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.962993 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-sg225"] Jul 02 13:42:47.000940 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:46.999959 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7"] Jul 02 13:42:47.029924 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:42:47.029487 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda783602a_9039_4d41_ad1f_13cedff176cf.slice/crio-19a7c501c20158cdbaf557cffb2b261fbe5d66504cd80e083f46f9b06fc47b9a WatchSource:0}: Error finding container 19a7c501c20158cdbaf557cffb2b261fbe5d66504cd80e083f46f9b06fc47b9a: Status 404 returned error can't find the container with id 19a7c501c20158cdbaf557cffb2b261fbe5d66504cd80e083f46f9b06fc47b9a Jul 02 13:42:47.135695 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.135633 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-kvknk"] Jul 02 13:42:47.160645 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.159912 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 13:42:47.220584 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.220546 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-864ff6486b-zhmhz"] Jul 02 13:42:47.235326 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:42:47.235291 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod716d8090_1589_4a66_9784_fdd03c5e4358.slice/crio-2a08a78cd2d2b3003b0ed007fec3f8a98c249d6dbbefbb8fb120d80f8d90a933 WatchSource:0}: Error finding container 2a08a78cd2d2b3003b0ed007fec3f8a98c249d6dbbefbb8fb120d80f8d90a933: Status 404 returned error can't find the container with id 2a08a78cd2d2b3003b0ed007fec3f8a98c249d6dbbefbb8fb120d80f8d90a933 Jul 02 13:42:47.255798 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.255759 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5896977f4b-hmj9w"] Jul 02 13:42:47.704698 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.704597 2561 generic.go:356] "Generic (PLEG): container finished" podID="15039e2a-095d-4e7a-96aa-45ab21a5b491" containerID="49f1b1a2135b74ab0f92a77ae48870ad902be26e7429fb9235e6517e3546d9d1" exitCode=0 Jul 02 13:42:47.704859 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.704699 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tp7m9" event={"ID":"15039e2a-095d-4e7a-96aa-45ab21a5b491","Type":"ContainerDied","Data":"49f1b1a2135b74ab0f92a77ae48870ad902be26e7429fb9235e6517e3546d9d1"} Jul 02 13:42:47.711629 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.711562 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" event={"ID":"62c2d08c-f0b1-4860-910f-48aab8a5101f","Type":"ContainerStarted","Data":"e6626fa81378da58612ef17bac5554f5a83c8bfdc5c575780a79586bea1df3bc"} Jul 02 13:42:47.711791 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.711664 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" event={"ID":"62c2d08c-f0b1-4860-910f-48aab8a5101f","Type":"ContainerStarted","Data":"8569329aeb4f9771a55e04b456e379a1f4ddaea7dedeec347a3f20c708766d14"} Jul 02 13:42:47.711791 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.711695 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" event={"ID":"62c2d08c-f0b1-4860-910f-48aab8a5101f","Type":"ContainerStarted","Data":"e64f107b2de789b849856a45b2bdc825c3f9154e6c64650a154fae18c57ca9fa"} Jul 02 13:42:47.716574 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.716541 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j9rh2" event={"ID":"da0298d6-5c20-459a-864b-0a3146e256ee","Type":"ContainerStarted","Data":"4e8bcd2566e06e427f591256f5ec5a3ad76e28c23696c7f10bbdf2437af95f40"} Jul 02 13:42:47.716719 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.716605 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-j9rh2" event={"ID":"da0298d6-5c20-459a-864b-0a3146e256ee","Type":"ContainerStarted","Data":"735247fc8e32b8ab55f0241e6c8c15f8ca4020489d2a1b5d3a3f1a1a4be9bde0"} Jul 02 13:42:47.719321 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.719288 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-sg225" event={"ID":"8e465295-97e6-498f-a211-e54e6bad7273","Type":"ContainerStarted","Data":"7412f2e4e684bd295f779b5fef51fa01bbe16f809d25888389f24338288ee5c7"} Jul 02 13:42:47.726829 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.726714 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" event={"ID":"a783602a-9039-4d41-ad1f-13cedff176cf","Type":"ContainerStarted","Data":"19a7c501c20158cdbaf557cffb2b261fbe5d66504cd80e083f46f9b06fc47b9a"} Jul 02 13:42:47.732357 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.732278 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" event={"ID":"716d8090-1589-4a66-9784-fdd03c5e4358","Type":"ContainerStarted","Data":"2a08a78cd2d2b3003b0ed007fec3f8a98c249d6dbbefbb8fb120d80f8d90a933"} Jul 02 13:42:47.735234 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.735119 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"834585dd-3bd9-4116-af0e-22bb1287a590","Type":"ContainerStarted","Data":"ebc605d9e6d84e7fa7d9201cbe15e27a3616dd9e972fe2264ad59b76c8aa91fb"} Jul 02 13:42:47.741424 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.741355 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kvknk" event={"ID":"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1","Type":"ContainerStarted","Data":"24d90e387f06fd336ec0b9aaf1944e5267988378bcc7e01a65e2d9afc39c40ff"} Jul 02 13:42:47.747028 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.746357 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jv42p" event={"ID":"684ebf83-fb83-47a1-94c7-ecd7bba444c7","Type":"ContainerStarted","Data":"83cdb07c178cdedc24fca1e69588e740e82858a60e7bffd5a2cd76b89ab3dd1d"} Jul 02 13:42:47.747028 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.746958 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:42:47.751154 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.751081 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5896977f4b-hmj9w" event={"ID":"ea824dc9-396f-445a-a895-d255dd7e5641","Type":"ContainerStarted","Data":"6f13b7e6466b684bbc3f4bf350b131e13157210ded2a8e1c56446693f1cb195b"} Jul 02 13:42:47.755461 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.755409 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-845b954455-8mr8c" podStartSLOduration=4.921222884 podStartE2EDuration="8.755350797s" totalImagesPullingTime="3.834127913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:42:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:42:47.752537855 +0000 UTC m=+73.878674363" watchObservedRunningTime="2026-07-02 13:42:47.755350797 +0000 UTC m=+73.881487305" Jul 02 13:42:47.756310 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.756025 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq" event={"ID":"4d282969-fbb0-425e-a612-9a5801ed1b30","Type":"ContainerStarted","Data":"caf6a806e82d806959281e91921e59d6fc33bc253a620a7cdfc960c294424937"} Jul 02 13:42:47.759701 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.759668 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-flv4b" event={"ID":"cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f","Type":"ContainerStarted","Data":"8ebfe395c2eb3a2e9730e08d205d412dd9af4174710f574c14dff793724143d4"} Jul 02 13:42:47.850995 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.848676 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-57f99c8b49-h6zdq" podStartSLOduration=3.392784329 podStartE2EDuration="8.84865298s" totalImagesPullingTime="5.455868651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:42:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:42:47.834876241 +0000 UTC m=+73.961012745" watchObservedRunningTime="2026-07-02 13:42:47.84865298 +0000 UTC m=+73.974789497" Jul 02 13:42:47.850995 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.848926 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-j9rh2" podStartSLOduration=70.859248695 podStartE2EDuration="1m13.848914916s" totalImagesPullingTime="2.989666221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:42:47.797515845 +0000 UTC m=+73.923652351" watchObservedRunningTime="2026-07-02 13:42:47.848914916 +0000 UTC m=+73.975051421" Jul 02 13:42:47.870381 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.870319 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-flv4b" podStartSLOduration=67.263823167 podStartE2EDuration="1m13.870296834s" totalImagesPullingTime="6.606473667s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:42:47.868229219 +0000 UTC m=+73.994366108" watchObservedRunningTime="2026-07-02 13:42:47.870296834 +0000 UTC m=+73.996433340" Jul 02 13:42:47.897267 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:47.896886 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jv42p" podStartSLOduration=67.766699566 podStartE2EDuration="1m13.896862056s" totalImagesPullingTime="6.13016249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:41:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:42:47.895785695 +0000 UTC m=+74.021922202" watchObservedRunningTime="2026-07-02 13:42:47.896862056 +0000 UTC m=+74.022998560" Jul 02 13:42:48.781874 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:48.781686 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tp7m9" event={"ID":"15039e2a-095d-4e7a-96aa-45ab21a5b491","Type":"ContainerStarted","Data":"2de4d455e0e513000e0ed08ec71f9d3d4b93e0d4b4b5386ef3803308b497b569"} Jul 02 13:42:48.781874 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:48.781761 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tp7m9" event={"ID":"15039e2a-095d-4e7a-96aa-45ab21a5b491","Type":"ContainerStarted","Data":"8876b404c73044bd047722777edc3b51852c20984f54a37ea812469c9f7e63e8"} Jul 02 13:42:48.788102 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:48.786344 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ef56f414-16ad-4597-83e3-46b3f1e93c15","Type":"ContainerStarted","Data":"859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d"} Jul 02 13:42:48.794944 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:48.794119 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"834585dd-3bd9-4116-af0e-22bb1287a590","Type":"ContainerStarted","Data":"d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566"} Jul 02 13:42:48.811493 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:48.811208 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tp7m9" podStartSLOduration=5.651065382 podStartE2EDuration="9.81117543s" totalImagesPullingTime="4.006367155s" totalInitContainerRuntime="153.742893ms" isStatefulPod=true podCreationTimestamp="2026-07-02 13:42:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:42:48.809469525 +0000 UTC m=+74.935606031" watchObservedRunningTime="2026-07-02 13:42:48.81117543 +0000 UTC m=+74.937311936" Jul 02 13:42:49.802633 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:49.802269 2561 generic.go:356] "Generic (PLEG): container finished" podID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerID="859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d" exitCode=0 Jul 02 13:42:49.802633 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:49.802360 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ef56f414-16ad-4597-83e3-46b3f1e93c15","Type":"ContainerDied","Data":"859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d"} Jul 02 13:42:49.805004 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:49.804934 2561 generic.go:356] "Generic (PLEG): container finished" podID="834585dd-3bd9-4116-af0e-22bb1287a590" containerID="d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566" exitCode=0 Jul 02 13:42:49.805125 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:49.805100 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"834585dd-3bd9-4116-af0e-22bb1287a590","Type":"ContainerDied","Data":"d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566"} Jul 02 13:42:52.592317 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.592277 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b79bbc56f-sm94p"] Jul 02 13:42:52.597327 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.597299 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.605765 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.605742 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 02 13:42:52.612711 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.612691 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b79bbc56f-sm94p"] Jul 02 13:42:52.729905 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.729868 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-console-config\") pod \"console-7b79bbc56f-sm94p\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.729905 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.729907 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-oauth-serving-cert\") pod \"console-7b79bbc56f-sm94p\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.730170 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.729937 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-service-ca\") pod \"console-7b79bbc56f-sm94p\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.730170 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.730020 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb83e970-4eb8-4518-bec4-e7127c3f21df-console-serving-cert\") pod \"console-7b79bbc56f-sm94p\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.730170 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.730063 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-trusted-ca-bundle\") pod \"console-7b79bbc56f-sm94p\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.730170 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.730108 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb83e970-4eb8-4518-bec4-e7127c3f21df-console-oauth-config\") pod \"console-7b79bbc56f-sm94p\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.730170 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.730133 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmhfq\" (UniqueName: \"kubernetes.io/projected/fb83e970-4eb8-4518-bec4-e7127c3f21df-kube-api-access-tmhfq\") pod \"console-7b79bbc56f-sm94p\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.831505 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.831466 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb83e970-4eb8-4518-bec4-e7127c3f21df-console-serving-cert\") pod \"console-7b79bbc56f-sm94p\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.831697 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.831527 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-trusted-ca-bundle\") pod \"console-7b79bbc56f-sm94p\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.831697 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.831574 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb83e970-4eb8-4518-bec4-e7127c3f21df-console-oauth-config\") pod \"console-7b79bbc56f-sm94p\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.831697 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.831593 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmhfq\" (UniqueName: \"kubernetes.io/projected/fb83e970-4eb8-4518-bec4-e7127c3f21df-kube-api-access-tmhfq\") pod \"console-7b79bbc56f-sm94p\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.831697 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.831639 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-console-config\") pod \"console-7b79bbc56f-sm94p\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.831697 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.831659 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-oauth-serving-cert\") pod \"console-7b79bbc56f-sm94p\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.831697 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.831691 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-service-ca\") pod \"console-7b79bbc56f-sm94p\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.832481 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.832451 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-service-ca\") pod \"console-7b79bbc56f-sm94p\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.832596 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.832573 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-trusted-ca-bundle\") pod \"console-7b79bbc56f-sm94p\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.832803 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.832757 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-console-config\") pod \"console-7b79bbc56f-sm94p\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.832935 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.832797 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-oauth-serving-cert\") pod \"console-7b79bbc56f-sm94p\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.834370 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.834348 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb83e970-4eb8-4518-bec4-e7127c3f21df-console-serving-cert\") pod \"console-7b79bbc56f-sm94p\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.834479 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.834426 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb83e970-4eb8-4518-bec4-e7127c3f21df-console-oauth-config\") pod \"console-7b79bbc56f-sm94p\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.842246 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.842224 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmhfq\" (UniqueName: \"kubernetes.io/projected/fb83e970-4eb8-4518-bec4-e7127c3f21df-kube-api-access-tmhfq\") pod \"console-7b79bbc56f-sm94p\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:42:52.908431 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:42:52.908340 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:43:04.269872 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:04.269765 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b79bbc56f-sm94p"] Jul 02 13:43:04.310640 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:43:04.310604 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb83e970_4eb8_4518_bec4_e7127c3f21df.slice/crio-04f3a68ee19f08b3de8887d073ce5a9f0ba811854e7cf84544b0a10843a485c2 WatchSource:0}: Error finding container 04f3a68ee19f08b3de8887d073ce5a9f0ba811854e7cf84544b0a10843a485c2: Status 404 returned error can't find the container with id 04f3a68ee19f08b3de8887d073ce5a9f0ba811854e7cf84544b0a10843a485c2 Jul 02 13:43:04.895158 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:04.895076 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b79bbc56f-sm94p" event={"ID":"fb83e970-4eb8-4518-bec4-e7127c3f21df","Type":"ContainerStarted","Data":"4d7616c3099c7e665e48f78e8e1e56deaeb619e2a3331d2acc88464299c0c73e"} Jul 02 13:43:04.895702 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:04.895400 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b79bbc56f-sm94p" event={"ID":"fb83e970-4eb8-4518-bec4-e7127c3f21df","Type":"ContainerStarted","Data":"04f3a68ee19f08b3de8887d073ce5a9f0ba811854e7cf84544b0a10843a485c2"} Jul 02 13:43:04.898144 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:04.898091 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-sg225" event={"ID":"8e465295-97e6-498f-a211-e54e6bad7273","Type":"ContainerStarted","Data":"aa872ad43778a8c46074a94da7dd801410a1afe273ab02eb584bb62ffea85ca1"} Jul 02 13:43:04.902927 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:04.902406 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" event={"ID":"a783602a-9039-4d41-ad1f-13cedff176cf","Type":"ContainerStarted","Data":"d843aec3ff804a3d6c8fc360ebb2dbfde8c37493277e240f4e788bd8d1ff602d"} Jul 02 13:43:04.902927 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:04.902484 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" event={"ID":"a783602a-9039-4d41-ad1f-13cedff176cf","Type":"ContainerStarted","Data":"52d14bbf2bad1da6221ca5accc5294cecedde83eb8520e015a629ac1fd772a04"} Jul 02 13:43:04.902927 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:04.902524 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" event={"ID":"a783602a-9039-4d41-ad1f-13cedff176cf","Type":"ContainerStarted","Data":"c5ff94a54cac48559301aa19250005da133e29a593d5039982ada740361208b1"} Jul 02 13:43:04.906898 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:04.906862 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ef56f414-16ad-4597-83e3-46b3f1e93c15","Type":"ContainerStarted","Data":"536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c"} Jul 02 13:43:04.907082 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:04.906945 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ef56f414-16ad-4597-83e3-46b3f1e93c15","Type":"ContainerStarted","Data":"f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f"} Jul 02 13:43:04.910919 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:04.910896 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" event={"ID":"716d8090-1589-4a66-9784-fdd03c5e4358","Type":"ContainerStarted","Data":"6a6dc6c0692f18d52c27a3a8649c5c7e617bde31364d8df381da6a6182e271f8"} Jul 02 13:43:04.914837 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:04.914659 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"834585dd-3bd9-4116-af0e-22bb1287a590","Type":"ContainerStarted","Data":"67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c"} Jul 02 13:43:04.914837 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:04.914732 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"834585dd-3bd9-4116-af0e-22bb1287a590","Type":"ContainerStarted","Data":"6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df"} Jul 02 13:43:04.916660 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:04.916601 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kvknk" event={"ID":"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1","Type":"ContainerStarted","Data":"222f7b5d099330a2cab21f1784212769b6552e9ea028e617767e2d534d027617"} Jul 02 13:43:04.919562 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:04.918991 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5896977f4b-hmj9w" event={"ID":"ea824dc9-396f-445a-a895-d255dd7e5641","Type":"ContainerStarted","Data":"ff2e27657b25280cd19e296010e5afd2aa08f26058784eeeb279a33a26a559cc"} Jul 02 13:43:04.924901 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:04.921501 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5499b6fb8f-j4blc" event={"ID":"c0d17ca3-aa6c-4dce-a610-b7c009989a2f","Type":"ContainerStarted","Data":"6b07a5445248fab4a8ab7aeaacab0169c92f30ebeb4d0083bf3e45e796e61fa2"} Jul 02 13:43:04.924901 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:04.922889 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5499b6fb8f-j4blc" Jul 02 13:43:04.927802 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:04.925989 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b79bbc56f-sm94p" podStartSLOduration=12.925955701 podStartE2EDuration="12.925955701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:42:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:43:04.925807931 +0000 UTC m=+91.051944437" watchObservedRunningTime="2026-07-02 13:43:04.925955701 +0000 UTC m=+91.052092233" Jul 02 13:43:04.934446 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:04.934401 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5499b6fb8f-j4blc" Jul 02 13:43:04.967049 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:04.966687 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" podStartSLOduration=3.877066396 podStartE2EDuration="20.966664567s" totalImagesPullingTime="17.089598171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:42:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:43:04.946303244 +0000 UTC m=+91.072439753" watchObservedRunningTime="2026-07-02 13:43:04.966664567 +0000 UTC m=+91.092801073" Jul 02 13:43:04.968157 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:04.968108 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-cd8656b8-sg225" podStartSLOduration=6.279120583 podStartE2EDuration="20.968096105s" totalImagesPullingTime="14.688975522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:42:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:43:04.966254223 +0000 UTC m=+91.092390729" watchObservedRunningTime="2026-07-02 13:43:04.968096105 +0000 UTC m=+91.094232642" Jul 02 13:43:05.012256 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:05.012198 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5499b6fb8f-j4blc" podStartSLOduration=2.165581728 podStartE2EDuration="29.012172171s" totalImagesPullingTime="26.846590443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:42:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:43:04.98588775 +0000 UTC m=+91.112024256" watchObservedRunningTime="2026-07-02 13:43:05.012172171 +0000 UTC m=+91.138308677" Jul 02 13:43:05.012895 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:05.012847 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5896977f4b-hmj9w" podStartSLOduration=4.904430057 podStartE2EDuration="22.012831517s" totalImagesPullingTime="17.10840146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:42:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:43:05.010323151 +0000 UTC m=+91.136459657" watchObservedRunningTime="2026-07-02 13:43:05.012831517 +0000 UTC m=+91.138968023" Jul 02 13:43:05.933373 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:05.933333 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ef56f414-16ad-4597-83e3-46b3f1e93c15","Type":"ContainerStarted","Data":"62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5"} Jul 02 13:43:05.933822 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:05.933429 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ef56f414-16ad-4597-83e3-46b3f1e93c15","Type":"ContainerStarted","Data":"0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e"} Jul 02 13:43:05.933822 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:05.933492 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ef56f414-16ad-4597-83e3-46b3f1e93c15","Type":"ContainerStarted","Data":"b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a"} Jul 02 13:43:05.949338 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:05.949287 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"834585dd-3bd9-4116-af0e-22bb1287a590","Type":"ContainerStarted","Data":"2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87"} Jul 02 13:43:05.949507 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:05.949405 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"834585dd-3bd9-4116-af0e-22bb1287a590","Type":"ContainerStarted","Data":"0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8"} Jul 02 13:43:05.949555 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:05.949528 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"834585dd-3bd9-4116-af0e-22bb1287a590","Type":"ContainerStarted","Data":"b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c"} Jul 02 13:43:05.949602 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:05.949576 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"834585dd-3bd9-4116-af0e-22bb1287a590","Type":"ContainerStarted","Data":"20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466"} Jul 02 13:43:05.950693 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:05.950667 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-sg225" Jul 02 13:43:05.957146 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:05.957118 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-sg225" Jul 02 13:43:05.980007 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:05.979841 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.063930142 podStartE2EDuration="19.979822027s" totalImagesPullingTime="15.862889354s" totalInitContainerRuntime="53.002531ms" isStatefulPod=false podCreationTimestamp="2026-07-02 13:42:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:43:05.978140316 +0000 UTC m=+92.104276837" watchObservedRunningTime="2026-07-02 13:43:05.979822027 +0000 UTC m=+92.105958545" Jul 02 13:43:06.582902 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:06.582851 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:43:06.959531 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:06.959492 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" event={"ID":"a783602a-9039-4d41-ad1f-13cedff176cf","Type":"ContainerStarted","Data":"7bae508acb98e96f528799a435c20ea2d7f8f1b8c4b4ab38aff3d51d3c4136a7"} Jul 02 13:43:06.959955 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:06.959573 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" event={"ID":"a783602a-9039-4d41-ad1f-13cedff176cf","Type":"ContainerStarted","Data":"358022280d254476ceb0339a699cfcb504e157ffc0b9237563ff554f12eb9819"} Jul 02 13:43:06.959955 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:06.959618 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" event={"ID":"a783602a-9039-4d41-ad1f-13cedff176cf","Type":"ContainerStarted","Data":"cf2d0437d7821f4dc84aaf4f9278c5f8ff3741013625ff7db5eec33cd064ef87"} Jul 02 13:43:06.963965 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:06.963871 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ef56f414-16ad-4597-83e3-46b3f1e93c15","Type":"ContainerStarted","Data":"6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c"} Jul 02 13:43:06.966198 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:06.966165 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kvknk" event={"ID":"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1","Type":"ContainerStarted","Data":"e738f9e0b9f9f694cd3691df000c25b4f796ae8c46ae8a7c000313254880eab5"} Jul 02 13:43:06.994495 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:06.993594 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=6.63963879 podStartE2EDuration="25.993568948s" totalImagesPullingTime="19.330957053s" totalInitContainerRuntime="22.973105ms" isStatefulPod=false podCreationTimestamp="2026-07-02 13:42:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:43:06.991938552 +0000 UTC m=+93.118075096" watchObservedRunningTime="2026-07-02 13:43:06.993568948 +0000 UTC m=+93.119705455" Jul 02 13:43:07.974709 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:07.974677 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:43:08.005257 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:08.005182 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" podStartSLOduration=7.969453112 podStartE2EDuration="26.005142628s" totalImagesPullingTime="18.035689516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:42:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:43:07.99782039 +0000 UTC m=+94.123956907" watchObservedRunningTime="2026-07-02 13:43:08.005142628 +0000 UTC m=+94.131279134" Jul 02 13:43:10.993052 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:10.993008 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-kvknk" event={"ID":"0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1","Type":"ContainerStarted","Data":"8575d6cf87ba0dfb8bdd7d7ea3ae21cd4804837e364b4846dccd5961b77c80c2"} Jul 02 13:43:11.024611 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:11.024550 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-kvknk" podStartSLOduration=12.491953382 podStartE2EDuration="35.024529535s" totalImagesPullingTime="22.532576153s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:42:36 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:43:11.022205702 +0000 UTC m=+97.148342247" watchObservedRunningTime="2026-07-02 13:43:11.024529535 +0000 UTC m=+97.150666042" Jul 02 13:43:12.909053 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:12.909006 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:43:12.909519 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:12.909115 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:43:12.914566 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:12.914542 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:43:13.007545 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:13.007463 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:43:13.053380 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:13.053346 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5896977f4b-hmj9w"] Jul 02 13:43:13.522653 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:13.522614 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:43:13.982879 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:13.982854 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5d4bbc8bd-bb7t7" Jul 02 13:43:19.807868 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:19.807839 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jv42p" Jul 02 13:43:24.594011 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:24.593940 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:43:24.594407 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:24.594048 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:43:24.599228 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:24.599203 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:43:25.059317 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:25.059279 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-864ff6486b-zhmhz" Jul 02 13:43:38.080161 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:38.080111 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5896977f4b-hmj9w" podUID="ea824dc9-396f-445a-a895-d255dd7e5641" containerName="console" containerID="cri-o://ff2e27657b25280cd19e296010e5afd2aa08f26058784eeeb279a33a26a559cc" gracePeriod=15 Jul 02 13:43:38.352932 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:38.352843 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:43:38.479733 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:38.479698 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea824dc9-396f-445a-a895-d255dd7e5641-console-config\") pod \"ea824dc9-396f-445a-a895-d255dd7e5641\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " Jul 02 13:43:38.479733 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:38.479738 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26f8c\" (UniqueName: \"kubernetes.io/projected/ea824dc9-396f-445a-a895-d255dd7e5641-kube-api-access-26f8c\") pod \"ea824dc9-396f-445a-a895-d255dd7e5641\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " Jul 02 13:43:38.480012 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:38.479764 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea824dc9-396f-445a-a895-d255dd7e5641-console-serving-cert\") pod \"ea824dc9-396f-445a-a895-d255dd7e5641\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " Jul 02 13:43:38.480012 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:38.479792 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea824dc9-396f-445a-a895-d255dd7e5641-console-oauth-config\") pod \"ea824dc9-396f-445a-a895-d255dd7e5641\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " Jul 02 13:43:38.480012 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:38.479850 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea824dc9-396f-445a-a895-d255dd7e5641-service-ca\") pod \"ea824dc9-396f-445a-a895-d255dd7e5641\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " Jul 02 13:43:38.480012 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:38.479906 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea824dc9-396f-445a-a895-d255dd7e5641-oauth-serving-cert\") pod \"ea824dc9-396f-445a-a895-d255dd7e5641\" (UID: \"ea824dc9-396f-445a-a895-d255dd7e5641\") " Jul 02 13:43:38.480297 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:38.480268 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea824dc9-396f-445a-a895-d255dd7e5641-console-config" (OuterVolumeSpecName: "console-config") pod "ea824dc9-396f-445a-a895-d255dd7e5641" (UID: "ea824dc9-396f-445a-a895-d255dd7e5641"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:43:38.480409 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:38.480270 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea824dc9-396f-445a-a895-d255dd7e5641-service-ca" (OuterVolumeSpecName: "service-ca") pod "ea824dc9-396f-445a-a895-d255dd7e5641" (UID: "ea824dc9-396f-445a-a895-d255dd7e5641"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:43:38.480409 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:38.480339 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea824dc9-396f-445a-a895-d255dd7e5641-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ea824dc9-396f-445a-a895-d255dd7e5641" (UID: "ea824dc9-396f-445a-a895-d255dd7e5641"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:43:38.482218 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:38.482195 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea824dc9-396f-445a-a895-d255dd7e5641-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ea824dc9-396f-445a-a895-d255dd7e5641" (UID: "ea824dc9-396f-445a-a895-d255dd7e5641"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:43:38.482525 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:38.482508 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea824dc9-396f-445a-a895-d255dd7e5641-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ea824dc9-396f-445a-a895-d255dd7e5641" (UID: "ea824dc9-396f-445a-a895-d255dd7e5641"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:43:38.482602 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:38.482552 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea824dc9-396f-445a-a895-d255dd7e5641-kube-api-access-26f8c" (OuterVolumeSpecName: "kube-api-access-26f8c") pod "ea824dc9-396f-445a-a895-d255dd7e5641" (UID: "ea824dc9-396f-445a-a895-d255dd7e5641"). InnerVolumeSpecName "kube-api-access-26f8c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:43:38.580790 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:38.580748 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea824dc9-396f-445a-a895-d255dd7e5641-console-config\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:43:38.580790 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:38.580781 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26f8c\" (UniqueName: \"kubernetes.io/projected/ea824dc9-396f-445a-a895-d255dd7e5641-kube-api-access-26f8c\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:43:38.580790 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:38.580792 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea824dc9-396f-445a-a895-d255dd7e5641-console-serving-cert\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:43:38.580790 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:38.580801 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea824dc9-396f-445a-a895-d255dd7e5641-console-oauth-config\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:43:38.581082 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:38.580809 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea824dc9-396f-445a-a895-d255dd7e5641-service-ca\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:43:38.581082 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:38.580818 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea824dc9-396f-445a-a895-d255dd7e5641-oauth-serving-cert\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:43:39.113633 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:39.113594 2561 generic.go:356] "Generic (PLEG): container finished" podID="ea824dc9-396f-445a-a895-d255dd7e5641" containerID="ff2e27657b25280cd19e296010e5afd2aa08f26058784eeeb279a33a26a559cc" exitCode=0 Jul 02 13:43:39.113633 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:39.113640 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5896977f4b-hmj9w" Jul 02 13:43:39.114160 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:39.113686 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5896977f4b-hmj9w" event={"ID":"ea824dc9-396f-445a-a895-d255dd7e5641","Type":"ContainerDied","Data":"ff2e27657b25280cd19e296010e5afd2aa08f26058784eeeb279a33a26a559cc"} Jul 02 13:43:39.114160 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:39.113746 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5896977f4b-hmj9w" event={"ID":"ea824dc9-396f-445a-a895-d255dd7e5641","Type":"ContainerDied","Data":"6f13b7e6466b684bbc3f4bf350b131e13157210ded2a8e1c56446693f1cb195b"} Jul 02 13:43:39.114160 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:39.113792 2561 scope.go:117] "RemoveContainer" containerID="ff2e27657b25280cd19e296010e5afd2aa08f26058784eeeb279a33a26a559cc" Jul 02 13:43:39.123212 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:39.123189 2561 scope.go:117] "RemoveContainer" containerID="ff2e27657b25280cd19e296010e5afd2aa08f26058784eeeb279a33a26a559cc" Jul 02 13:43:39.123546 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:43:39.123525 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff2e27657b25280cd19e296010e5afd2aa08f26058784eeeb279a33a26a559cc\": container with ID starting with ff2e27657b25280cd19e296010e5afd2aa08f26058784eeeb279a33a26a559cc not found: ID does not exist" containerID="ff2e27657b25280cd19e296010e5afd2aa08f26058784eeeb279a33a26a559cc" Jul 02 13:43:39.123595 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:39.123555 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff2e27657b25280cd19e296010e5afd2aa08f26058784eeeb279a33a26a559cc"} err="failed to get container status \"ff2e27657b25280cd19e296010e5afd2aa08f26058784eeeb279a33a26a559cc\": rpc error: code = NotFound desc = could not find container \"ff2e27657b25280cd19e296010e5afd2aa08f26058784eeeb279a33a26a559cc\": container with ID starting with ff2e27657b25280cd19e296010e5afd2aa08f26058784eeeb279a33a26a559cc not found: ID does not exist" Jul 02 13:43:39.133732 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:39.133699 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5896977f4b-hmj9w"] Jul 02 13:43:39.138040 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:39.138005 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5896977f4b-hmj9w"] Jul 02 13:43:40.362736 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:40.362702 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea824dc9-396f-445a-a895-d255dd7e5641" path="/var/lib/kubelet/pods/ea824dc9-396f-445a-a895-d255dd7e5641/volumes" Jul 02 13:43:46.583047 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:46.583005 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:43:46.648987 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:46.648950 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:43:47.163775 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:43:47.163742 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:00.274010 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.273960 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 13:44:00.274411 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.274346 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="alertmanager" containerID="cri-o://f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f" gracePeriod=120 Jul 02 13:44:00.274481 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.274417 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="kube-rbac-proxy-metric" containerID="cri-o://62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5" gracePeriod=120 Jul 02 13:44:00.274536 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.274457 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="prom-label-proxy" containerID="cri-o://6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c" gracePeriod=120 Jul 02 13:44:00.274536 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.274467 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="kube-rbac-proxy-web" containerID="cri-o://b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a" gracePeriod=120 Jul 02 13:44:00.274536 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.274495 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="kube-rbac-proxy" containerID="cri-o://0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e" gracePeriod=120 Jul 02 13:44:00.274681 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.274641 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="config-reloader" containerID="cri-o://536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c" gracePeriod=120 Jul 02 13:44:00.532998 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.532926 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:00.576291 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.576260 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-config-volume\") pod \"ef56f414-16ad-4597-83e3-46b3f1e93c15\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " Jul 02 13:44:00.576291 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.576303 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef56f414-16ad-4597-83e3-46b3f1e93c15-config-out\") pod \"ef56f414-16ad-4597-83e3-46b3f1e93c15\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " Jul 02 13:44:00.576531 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.576337 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef56f414-16ad-4597-83e3-46b3f1e93c15-alertmanager-trusted-ca-bundle\") pod \"ef56f414-16ad-4597-83e3-46b3f1e93c15\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " Jul 02 13:44:00.576531 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.576359 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-cluster-tls-config\") pod \"ef56f414-16ad-4597-83e3-46b3f1e93c15\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " Jul 02 13:44:00.576531 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.576387 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-main-tls\") pod \"ef56f414-16ad-4597-83e3-46b3f1e93c15\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " Jul 02 13:44:00.576531 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.576433 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwjtb\" (UniqueName: \"kubernetes.io/projected/ef56f414-16ad-4597-83e3-46b3f1e93c15-kube-api-access-wwjtb\") pod \"ef56f414-16ad-4597-83e3-46b3f1e93c15\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " Jul 02 13:44:00.576531 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.576514 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef56f414-16ad-4597-83e3-46b3f1e93c15-tls-assets\") pod \"ef56f414-16ad-4597-83e3-46b3f1e93c15\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " Jul 02 13:44:00.576789 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.576544 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ef56f414-16ad-4597-83e3-46b3f1e93c15-alertmanager-main-db\") pod \"ef56f414-16ad-4597-83e3-46b3f1e93c15\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " Jul 02 13:44:00.576789 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.576569 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-kube-rbac-proxy-web\") pod \"ef56f414-16ad-4597-83e3-46b3f1e93c15\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " Jul 02 13:44:00.576789 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.576597 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-web-config\") pod \"ef56f414-16ad-4597-83e3-46b3f1e93c15\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " Jul 02 13:44:00.576789 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.576630 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-kube-rbac-proxy\") pod \"ef56f414-16ad-4597-83e3-46b3f1e93c15\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " Jul 02 13:44:00.576789 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.576657 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-kube-rbac-proxy-metric\") pod \"ef56f414-16ad-4597-83e3-46b3f1e93c15\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " Jul 02 13:44:00.579116 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.577104 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef56f414-16ad-4597-83e3-46b3f1e93c15-metrics-client-ca\") pod \"ef56f414-16ad-4597-83e3-46b3f1e93c15\" (UID: \"ef56f414-16ad-4597-83e3-46b3f1e93c15\") " Jul 02 13:44:00.579116 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.576866 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef56f414-16ad-4597-83e3-46b3f1e93c15-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "ef56f414-16ad-4597-83e3-46b3f1e93c15" (UID: "ef56f414-16ad-4597-83e3-46b3f1e93c15"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:44:00.579116 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.577852 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef56f414-16ad-4597-83e3-46b3f1e93c15-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "ef56f414-16ad-4597-83e3-46b3f1e93c15" (UID: "ef56f414-16ad-4597-83e3-46b3f1e93c15"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 13:44:00.579116 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.578223 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef56f414-16ad-4597-83e3-46b3f1e93c15-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "ef56f414-16ad-4597-83e3-46b3f1e93c15" (UID: "ef56f414-16ad-4597-83e3-46b3f1e93c15"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:44:00.579832 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.579765 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef56f414-16ad-4597-83e3-46b3f1e93c15-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ef56f414-16ad-4597-83e3-46b3f1e93c15" (UID: "ef56f414-16ad-4597-83e3-46b3f1e93c15"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:44:00.580296 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.580171 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "ef56f414-16ad-4597-83e3-46b3f1e93c15" (UID: "ef56f414-16ad-4597-83e3-46b3f1e93c15"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:44:00.580296 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.580263 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef56f414-16ad-4597-83e3-46b3f1e93c15-config-out" (OuterVolumeSpecName: "config-out") pod "ef56f414-16ad-4597-83e3-46b3f1e93c15" (UID: "ef56f414-16ad-4597-83e3-46b3f1e93c15"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 13:44:00.582202 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.582150 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef56f414-16ad-4597-83e3-46b3f1e93c15-kube-api-access-wwjtb" (OuterVolumeSpecName: "kube-api-access-wwjtb") pod "ef56f414-16ad-4597-83e3-46b3f1e93c15" (UID: "ef56f414-16ad-4597-83e3-46b3f1e93c15"). InnerVolumeSpecName "kube-api-access-wwjtb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:44:00.582399 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.582358 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "ef56f414-16ad-4597-83e3-46b3f1e93c15" (UID: "ef56f414-16ad-4597-83e3-46b3f1e93c15"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:44:00.582478 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.582453 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-config-volume" (OuterVolumeSpecName: "config-volume") pod "ef56f414-16ad-4597-83e3-46b3f1e93c15" (UID: "ef56f414-16ad-4597-83e3-46b3f1e93c15"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:44:00.582616 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.582596 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "ef56f414-16ad-4597-83e3-46b3f1e93c15" (UID: "ef56f414-16ad-4597-83e3-46b3f1e93c15"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:44:00.584024 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.583940 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "ef56f414-16ad-4597-83e3-46b3f1e93c15" (UID: "ef56f414-16ad-4597-83e3-46b3f1e93c15"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:44:00.586443 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.586417 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "ef56f414-16ad-4597-83e3-46b3f1e93c15" (UID: "ef56f414-16ad-4597-83e3-46b3f1e93c15"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:44:00.594386 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.594348 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-web-config" (OuterVolumeSpecName: "web-config") pod "ef56f414-16ad-4597-83e3-46b3f1e93c15" (UID: "ef56f414-16ad-4597-83e3-46b3f1e93c15"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:44:00.678345 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.678303 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwjtb\" (UniqueName: \"kubernetes.io/projected/ef56f414-16ad-4597-83e3-46b3f1e93c15-kube-api-access-wwjtb\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:00.678345 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.678335 2561 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ef56f414-16ad-4597-83e3-46b3f1e93c15-tls-assets\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:00.678345 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.678344 2561 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ef56f414-16ad-4597-83e3-46b3f1e93c15-alertmanager-main-db\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:00.678345 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.678355 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:00.678607 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.678365 2561 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-web-config\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:00.678607 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.678374 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:00.678607 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.678383 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:00.678607 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.678392 2561 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef56f414-16ad-4597-83e3-46b3f1e93c15-metrics-client-ca\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:00.678607 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.678401 2561 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-config-volume\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:00.678607 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.678409 2561 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ef56f414-16ad-4597-83e3-46b3f1e93c15-config-out\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:00.678607 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.678419 2561 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef56f414-16ad-4597-83e3-46b3f1e93c15-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:00.678607 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.678428 2561 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-cluster-tls-config\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:00.678607 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:00.678436 2561 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ef56f414-16ad-4597-83e3-46b3f1e93c15-secret-alertmanager-main-tls\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:01.207533 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.207499 2561 generic.go:356] "Generic (PLEG): container finished" podID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerID="6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c" exitCode=0 Jul 02 13:44:01.207533 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.207524 2561 generic.go:356] "Generic (PLEG): container finished" podID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerID="62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5" exitCode=0 Jul 02 13:44:01.207533 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.207531 2561 generic.go:356] "Generic (PLEG): container finished" podID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerID="0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e" exitCode=0 Jul 02 13:44:01.207533 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.207537 2561 generic.go:356] "Generic (PLEG): container finished" podID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerID="b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a" exitCode=0 Jul 02 13:44:01.207533 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.207542 2561 generic.go:356] "Generic (PLEG): container finished" podID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerID="536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c" exitCode=0 Jul 02 13:44:01.207932 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.207548 2561 generic.go:356] "Generic (PLEG): container finished" podID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerID="f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f" exitCode=0 Jul 02 13:44:01.207932 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.207575 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ef56f414-16ad-4597-83e3-46b3f1e93c15","Type":"ContainerDied","Data":"6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c"} Jul 02 13:44:01.207932 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.207646 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ef56f414-16ad-4597-83e3-46b3f1e93c15","Type":"ContainerDied","Data":"62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5"} Jul 02 13:44:01.207932 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.207587 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.207932 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.207675 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ef56f414-16ad-4597-83e3-46b3f1e93c15","Type":"ContainerDied","Data":"0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e"} Jul 02 13:44:01.207932 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.207700 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ef56f414-16ad-4597-83e3-46b3f1e93c15","Type":"ContainerDied","Data":"b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a"} Jul 02 13:44:01.207932 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.207725 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ef56f414-16ad-4597-83e3-46b3f1e93c15","Type":"ContainerDied","Data":"536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c"} Jul 02 13:44:01.207932 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.207748 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ef56f414-16ad-4597-83e3-46b3f1e93c15","Type":"ContainerDied","Data":"f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f"} Jul 02 13:44:01.207932 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.207754 2561 scope.go:117] "RemoveContainer" containerID="6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c" Jul 02 13:44:01.207932 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.207772 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ef56f414-16ad-4597-83e3-46b3f1e93c15","Type":"ContainerDied","Data":"23b4df46cb4a7b89fc6a0f73b58e22afa6d7c1f1704e0c90915e930785718a21"} Jul 02 13:44:01.216747 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.216648 2561 scope.go:117] "RemoveContainer" containerID="62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5" Jul 02 13:44:01.224638 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.224618 2561 scope.go:117] "RemoveContainer" containerID="0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e" Jul 02 13:44:01.232731 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.232649 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 13:44:01.233641 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.233621 2561 scope.go:117] "RemoveContainer" containerID="b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a" Jul 02 13:44:01.237106 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.237080 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 13:44:01.242382 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.242359 2561 scope.go:117] "RemoveContainer" containerID="536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c" Jul 02 13:44:01.251747 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.251729 2561 scope.go:117] "RemoveContainer" containerID="f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f" Jul 02 13:44:01.259519 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.259492 2561 scope.go:117] "RemoveContainer" containerID="859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d" Jul 02 13:44:01.263322 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263298 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 13:44:01.263636 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263624 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="kube-rbac-proxy-web" Jul 02 13:44:01.263684 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263639 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="kube-rbac-proxy-web" Jul 02 13:44:01.263684 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263650 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="alertmanager" Jul 02 13:44:01.263684 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263655 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="alertmanager" Jul 02 13:44:01.263684 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263663 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="config-reloader" Jul 02 13:44:01.263684 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263668 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="config-reloader" Jul 02 13:44:01.263684 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263678 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea824dc9-396f-445a-a895-d255dd7e5641" containerName="console" Jul 02 13:44:01.263850 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263690 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea824dc9-396f-445a-a895-d255dd7e5641" containerName="console" Jul 02 13:44:01.263850 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263700 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="init-config-reloader" Jul 02 13:44:01.263850 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263706 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="init-config-reloader" Jul 02 13:44:01.263850 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263714 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="kube-rbac-proxy-metric" Jul 02 13:44:01.263850 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263719 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="kube-rbac-proxy-metric" Jul 02 13:44:01.263850 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263728 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="prom-label-proxy" Jul 02 13:44:01.263850 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263733 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="prom-label-proxy" Jul 02 13:44:01.263850 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263739 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="kube-rbac-proxy" Jul 02 13:44:01.263850 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263743 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="kube-rbac-proxy" Jul 02 13:44:01.263850 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263800 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="kube-rbac-proxy" Jul 02 13:44:01.263850 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263808 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="prom-label-proxy" Jul 02 13:44:01.263850 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263815 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea824dc9-396f-445a-a895-d255dd7e5641" containerName="console" Jul 02 13:44:01.263850 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263820 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="kube-rbac-proxy-web" Jul 02 13:44:01.263850 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263826 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="kube-rbac-proxy-metric" Jul 02 13:44:01.263850 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263832 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="config-reloader" Jul 02 13:44:01.263850 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.263840 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" containerName="alertmanager" Jul 02 13:44:01.267763 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.267750 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.267822 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.267798 2561 scope.go:117] "RemoveContainer" containerID="6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c" Jul 02 13:44:01.268171 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:44:01.268140 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c\": container with ID starting with 6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c not found: ID does not exist" containerID="6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c" Jul 02 13:44:01.268254 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.268180 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c"} err="failed to get container status \"6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c\": rpc error: code = NotFound desc = could not find container \"6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c\": container with ID starting with 6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c not found: ID does not exist" Jul 02 13:44:01.268254 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.268198 2561 scope.go:117] "RemoveContainer" containerID="62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5" Jul 02 13:44:01.268533 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:44:01.268513 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5\": container with ID starting with 62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5 not found: ID does not exist" containerID="62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5" Jul 02 13:44:01.268589 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.268540 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5"} err="failed to get container status \"62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5\": rpc error: code = NotFound desc = could not find container \"62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5\": container with ID starting with 62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5 not found: ID does not exist" Jul 02 13:44:01.268589 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.268555 2561 scope.go:117] "RemoveContainer" containerID="0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e" Jul 02 13:44:01.268809 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:44:01.268794 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e\": container with ID starting with 0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e not found: ID does not exist" containerID="0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e" Jul 02 13:44:01.268849 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.268816 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e"} err="failed to get container status \"0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e\": rpc error: code = NotFound desc = could not find container \"0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e\": container with ID starting with 0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e not found: ID does not exist" Jul 02 13:44:01.268849 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.268832 2561 scope.go:117] "RemoveContainer" containerID="b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a" Jul 02 13:44:01.269180 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:44:01.269162 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a\": container with ID starting with b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a not found: ID does not exist" containerID="b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a" Jul 02 13:44:01.269240 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.269185 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a"} err="failed to get container status \"b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a\": rpc error: code = NotFound desc = could not find container \"b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a\": container with ID starting with b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a not found: ID does not exist" Jul 02 13:44:01.269240 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.269199 2561 scope.go:117] "RemoveContainer" containerID="536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c" Jul 02 13:44:01.269444 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:44:01.269428 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c\": container with ID starting with 536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c not found: ID does not exist" containerID="536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c" Jul 02 13:44:01.269494 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.269447 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c"} err="failed to get container status \"536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c\": rpc error: code = NotFound desc = could not find container \"536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c\": container with ID starting with 536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c not found: ID does not exist" Jul 02 13:44:01.269494 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.269460 2561 scope.go:117] "RemoveContainer" containerID="f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f" Jul 02 13:44:01.269702 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:44:01.269689 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f\": container with ID starting with f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f not found: ID does not exist" containerID="f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f" Jul 02 13:44:01.269752 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.269704 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f"} err="failed to get container status \"f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f\": rpc error: code = NotFound desc = could not find container \"f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f\": container with ID starting with f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f not found: ID does not exist" Jul 02 13:44:01.269752 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.269715 2561 scope.go:117] "RemoveContainer" containerID="859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d" Jul 02 13:44:01.269913 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:44:01.269894 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d\": container with ID starting with 859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d not found: ID does not exist" containerID="859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d" Jul 02 13:44:01.269956 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.269919 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d"} err="failed to get container status \"859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d\": rpc error: code = NotFound desc = could not find container \"859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d\": container with ID starting with 859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d not found: ID does not exist" Jul 02 13:44:01.269956 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.269939 2561 scope.go:117] "RemoveContainer" containerID="6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c" Jul 02 13:44:01.270200 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.270178 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c"} err="failed to get container status \"6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c\": rpc error: code = NotFound desc = could not find container \"6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c\": container with ID starting with 6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c not found: ID does not exist" Jul 02 13:44:01.270265 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.270202 2561 scope.go:117] "RemoveContainer" containerID="62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5" Jul 02 13:44:01.270265 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.270214 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 02 13:44:01.270265 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.270225 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 02 13:44:01.270265 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.270191 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 02 13:44:01.270448 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.270270 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-hmb8n\"" Jul 02 13:44:01.270578 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.270541 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5"} err="failed to get container status \"62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5\": rpc error: code = NotFound desc = could not find container \"62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5\": container with ID starting with 62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5 not found: ID does not exist" Jul 02 13:44:01.270657 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.270592 2561 scope.go:117] "RemoveContainer" containerID="0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e" Jul 02 13:44:01.270657 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.270633 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 02 13:44:01.270763 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.270646 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 02 13:44:01.270804 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.270766 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 02 13:44:01.270857 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.270803 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 02 13:44:01.270857 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.270808 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 02 13:44:01.271147 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.271118 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e"} err="failed to get container status \"0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e\": rpc error: code = NotFound desc = could not find container \"0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e\": container with ID starting with 0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e not found: ID does not exist" Jul 02 13:44:01.271147 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.271148 2561 scope.go:117] "RemoveContainer" containerID="b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a" Jul 02 13:44:01.273421 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.273394 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a"} err="failed to get container status \"b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a\": rpc error: code = NotFound desc = could not find container \"b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a\": container with ID starting with b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a not found: ID does not exist" Jul 02 13:44:01.273421 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.273422 2561 scope.go:117] "RemoveContainer" containerID="536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c" Jul 02 13:44:01.273719 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.273696 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c"} err="failed to get container status \"536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c\": rpc error: code = NotFound desc = could not find container \"536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c\": container with ID starting with 536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c not found: ID does not exist" Jul 02 13:44:01.273818 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.273720 2561 scope.go:117] "RemoveContainer" containerID="f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f" Jul 02 13:44:01.274038 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.274019 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f"} err="failed to get container status \"f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f\": rpc error: code = NotFound desc = could not find container \"f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f\": container with ID starting with f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f not found: ID does not exist" Jul 02 13:44:01.274038 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.274037 2561 scope.go:117] "RemoveContainer" containerID="859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d" Jul 02 13:44:01.274370 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.274319 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d"} err="failed to get container status \"859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d\": rpc error: code = NotFound desc = could not find container \"859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d\": container with ID starting with 859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d not found: ID does not exist" Jul 02 13:44:01.274420 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.274370 2561 scope.go:117] "RemoveContainer" containerID="6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c" Jul 02 13:44:01.274643 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.274618 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c"} err="failed to get container status \"6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c\": rpc error: code = NotFound desc = could not find container \"6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c\": container with ID starting with 6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c not found: ID does not exist" Jul 02 13:44:01.274715 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.274644 2561 scope.go:117] "RemoveContainer" containerID="62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5" Jul 02 13:44:01.274881 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.274853 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5"} err="failed to get container status \"62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5\": rpc error: code = NotFound desc = could not find container \"62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5\": container with ID starting with 62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5 not found: ID does not exist" Jul 02 13:44:01.274881 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.274879 2561 scope.go:117] "RemoveContainer" containerID="0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e" Jul 02 13:44:01.275120 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.275094 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e"} err="failed to get container status \"0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e\": rpc error: code = NotFound desc = could not find container \"0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e\": container with ID starting with 0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e not found: ID does not exist" Jul 02 13:44:01.275120 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.275119 2561 scope.go:117] "RemoveContainer" containerID="b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a" Jul 02 13:44:01.275385 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.275362 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a"} err="failed to get container status \"b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a\": rpc error: code = NotFound desc = could not find container \"b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a\": container with ID starting with b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a not found: ID does not exist" Jul 02 13:44:01.275385 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.275386 2561 scope.go:117] "RemoveContainer" containerID="536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c" Jul 02 13:44:01.275683 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.275644 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c"} err="failed to get container status \"536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c\": rpc error: code = NotFound desc = could not find container \"536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c\": container with ID starting with 536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c not found: ID does not exist" Jul 02 13:44:01.275683 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.275679 2561 scope.go:117] "RemoveContainer" containerID="f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f" Jul 02 13:44:01.275960 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.275938 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f"} err="failed to get container status \"f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f\": rpc error: code = NotFound desc = could not find container \"f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f\": container with ID starting with f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f not found: ID does not exist" Jul 02 13:44:01.276051 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.275963 2561 scope.go:117] "RemoveContainer" containerID="859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d" Jul 02 13:44:01.276250 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.276223 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d"} err="failed to get container status \"859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d\": rpc error: code = NotFound desc = could not find container \"859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d\": container with ID starting with 859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d not found: ID does not exist" Jul 02 13:44:01.276250 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.276250 2561 scope.go:117] "RemoveContainer" containerID="6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c" Jul 02 13:44:01.276515 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.276491 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c"} err="failed to get container status \"6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c\": rpc error: code = NotFound desc = could not find container \"6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c\": container with ID starting with 6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c not found: ID does not exist" Jul 02 13:44:01.276515 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.276516 2561 scope.go:117] "RemoveContainer" containerID="62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5" Jul 02 13:44:01.276856 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.276781 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5"} err="failed to get container status \"62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5\": rpc error: code = NotFound desc = could not find container \"62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5\": container with ID starting with 62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5 not found: ID does not exist" Jul 02 13:44:01.276856 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.276803 2561 scope.go:117] "RemoveContainer" containerID="0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e" Jul 02 13:44:01.277121 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.277083 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 02 13:44:01.277250 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.277228 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e"} err="failed to get container status \"0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e\": rpc error: code = NotFound desc = could not find container \"0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e\": container with ID starting with 0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e not found: ID does not exist" Jul 02 13:44:01.277311 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.277255 2561 scope.go:117] "RemoveContainer" containerID="b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a" Jul 02 13:44:01.277512 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.277491 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a"} err="failed to get container status \"b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a\": rpc error: code = NotFound desc = could not find container \"b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a\": container with ID starting with b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a not found: ID does not exist" Jul 02 13:44:01.277561 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.277515 2561 scope.go:117] "RemoveContainer" containerID="536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c" Jul 02 13:44:01.277738 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.277721 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c"} err="failed to get container status \"536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c\": rpc error: code = NotFound desc = could not find container \"536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c\": container with ID starting with 536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c not found: ID does not exist" Jul 02 13:44:01.277784 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.277739 2561 scope.go:117] "RemoveContainer" containerID="f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f" Jul 02 13:44:01.278415 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.278325 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f"} err="failed to get container status \"f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f\": rpc error: code = NotFound desc = could not find container \"f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f\": container with ID starting with f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f not found: ID does not exist" Jul 02 13:44:01.278524 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.278418 2561 scope.go:117] "RemoveContainer" containerID="859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d" Jul 02 13:44:01.278814 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.278786 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d"} err="failed to get container status \"859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d\": rpc error: code = NotFound desc = could not find container \"859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d\": container with ID starting with 859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d not found: ID does not exist" Jul 02 13:44:01.278899 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.278818 2561 scope.go:117] "RemoveContainer" containerID="6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c" Jul 02 13:44:01.279246 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.279166 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c"} err="failed to get container status \"6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c\": rpc error: code = NotFound desc = could not find container \"6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c\": container with ID starting with 6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c not found: ID does not exist" Jul 02 13:44:01.279246 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.279194 2561 scope.go:117] "RemoveContainer" containerID="62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5" Jul 02 13:44:01.279837 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.279634 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5"} err="failed to get container status \"62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5\": rpc error: code = NotFound desc = could not find container \"62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5\": container with ID starting with 62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5 not found: ID does not exist" Jul 02 13:44:01.279837 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.279660 2561 scope.go:117] "RemoveContainer" containerID="0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e" Jul 02 13:44:01.280026 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.279999 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e"} err="failed to get container status \"0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e\": rpc error: code = NotFound desc = could not find container \"0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e\": container with ID starting with 0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e not found: ID does not exist" Jul 02 13:44:01.280084 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.280028 2561 scope.go:117] "RemoveContainer" containerID="b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a" Jul 02 13:44:01.285527 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.285314 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a"} err="failed to get container status \"b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a\": rpc error: code = NotFound desc = could not find container \"b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a\": container with ID starting with b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a not found: ID does not exist" Jul 02 13:44:01.285527 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.285344 2561 scope.go:117] "RemoveContainer" containerID="536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c" Jul 02 13:44:01.286230 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.286204 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c"} err="failed to get container status \"536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c\": rpc error: code = NotFound desc = could not find container \"536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c\": container with ID starting with 536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c not found: ID does not exist" Jul 02 13:44:01.286331 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.286232 2561 scope.go:117] "RemoveContainer" containerID="f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f" Jul 02 13:44:01.286454 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.286433 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-config-out\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.286533 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.286476 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.286588 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.286552 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.286642 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.286601 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-config-volume\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.286642 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.286633 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.286733 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.286704 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.286781 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.286724 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f"} err="failed to get container status \"f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f\": rpc error: code = NotFound desc = could not find container \"f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f\": container with ID starting with f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f not found: ID does not exist" Jul 02 13:44:01.286781 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.286746 2561 scope.go:117] "RemoveContainer" containerID="859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d" Jul 02 13:44:01.286781 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.286760 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.287193 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.286800 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.287193 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.286903 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-web-config\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.287193 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.287077 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d"} err="failed to get container status \"859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d\": rpc error: code = NotFound desc = could not find container \"859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d\": container with ID starting with 859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d not found: ID does not exist" Jul 02 13:44:01.287193 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.287102 2561 scope.go:117] "RemoveContainer" containerID="6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c" Jul 02 13:44:01.287546 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.287242 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.287546 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.287287 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 13:44:01.287546 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.287301 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.287546 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.287350 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzdw4\" (UniqueName: \"kubernetes.io/projected/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-kube-api-access-vzdw4\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.287546 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.287394 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.287546 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.287492 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c"} err="failed to get container status \"6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c\": rpc error: code = NotFound desc = could not find container \"6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c\": container with ID starting with 6b8805b21990f927cba183e0467a6e76a7c8d318d236c57934d949878ec37b4c not found: ID does not exist" Jul 02 13:44:01.287546 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.287509 2561 scope.go:117] "RemoveContainer" containerID="62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5" Jul 02 13:44:01.288532 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.288453 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5"} err="failed to get container status \"62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5\": rpc error: code = NotFound desc = could not find container \"62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5\": container with ID starting with 62d4553e4f03812e656e8ad5f52731bee14e3f06b43390122d04acd4e41ebaf5 not found: ID does not exist" Jul 02 13:44:01.288605 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.288534 2561 scope.go:117] "RemoveContainer" containerID="0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e" Jul 02 13:44:01.288936 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.288910 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e"} err="failed to get container status \"0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e\": rpc error: code = NotFound desc = could not find container \"0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e\": container with ID starting with 0ef009b04d5f85ce16df3d22b6ee7401dae730e3e8a44d3ee9045df350f84e3e not found: ID does not exist" Jul 02 13:44:01.289071 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.288938 2561 scope.go:117] "RemoveContainer" containerID="b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a" Jul 02 13:44:01.289298 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.289273 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a"} err="failed to get container status \"b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a\": rpc error: code = NotFound desc = could not find container \"b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a\": container with ID starting with b9a9146235aa29cbfe824c95bf0eb3edd9ffb1a6d037167453cbe2f685d5db5a not found: ID does not exist" Jul 02 13:44:01.289370 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.289301 2561 scope.go:117] "RemoveContainer" containerID="536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c" Jul 02 13:44:01.289618 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.289594 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c"} err="failed to get container status \"536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c\": rpc error: code = NotFound desc = could not find container \"536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c\": container with ID starting with 536bda93034199b69ffa8bf77abb9051e1f2f0a02c604fe3159efe0f94f9621c not found: ID does not exist" Jul 02 13:44:01.289682 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.289620 2561 scope.go:117] "RemoveContainer" containerID="f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f" Jul 02 13:44:01.289872 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.289855 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f"} err="failed to get container status \"f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f\": rpc error: code = NotFound desc = could not find container \"f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f\": container with ID starting with f656a2aba233204dbf1120ff722ede7264b743d95df2df0d7c5fec5c49bce70f not found: ID does not exist" Jul 02 13:44:01.289940 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.289873 2561 scope.go:117] "RemoveContainer" containerID="859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d" Jul 02 13:44:01.290132 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.290115 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d"} err="failed to get container status \"859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d\": rpc error: code = NotFound desc = could not find container \"859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d\": container with ID starting with 859aa79a37b03930935e1cdc8d091383d1f8b6a1c61f985af085d7279e67418d not found: ID does not exist" Jul 02 13:44:01.388249 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.388204 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-web-config\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.388249 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.388253 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.388512 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.388269 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.388512 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.388292 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzdw4\" (UniqueName: \"kubernetes.io/projected/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-kube-api-access-vzdw4\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.388512 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.388313 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.388512 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.388338 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-config-out\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.388512 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.388359 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.388512 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.388389 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.388512 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.388417 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-config-volume\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.388512 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.388438 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.388512 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.388469 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.388512 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.388498 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.389075 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.388539 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.389075 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.389018 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.389782 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.389362 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.390018 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.389967 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.391342 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.391317 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.391463 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.391358 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.391616 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.391586 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-config-out\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.391745 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.391728 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-web-config\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.391907 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.391879 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.392370 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.392349 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.392463 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.392349 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.392775 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.392759 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-config-volume\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.393079 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.393062 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.397265 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.397242 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzdw4\" (UniqueName: \"kubernetes.io/projected/58dfdeb1-5afc-4881-8dd2-3ee2964032c2-kube-api-access-vzdw4\") pod \"alertmanager-main-0\" (UID: \"58dfdeb1-5afc-4881-8dd2-3ee2964032c2\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.581083 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.580985 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 13:44:01.723856 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:01.723825 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 13:44:01.726909 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:44:01.726884 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58dfdeb1_5afc_4881_8dd2_3ee2964032c2.slice/crio-82d71341fdc55fb8be76723b946333c5689e01baa38997d67678b07bfde36523 WatchSource:0}: Error finding container 82d71341fdc55fb8be76723b946333c5689e01baa38997d67678b07bfde36523: Status 404 returned error can't find the container with id 82d71341fdc55fb8be76723b946333c5689e01baa38997d67678b07bfde36523 Jul 02 13:44:02.213319 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:02.213280 2561 generic.go:356] "Generic (PLEG): container finished" podID="58dfdeb1-5afc-4881-8dd2-3ee2964032c2" containerID="3cd66691a3e98e7f52fac71496dffc840a323b8828cb15cb8eb387458295d6fd" exitCode=0 Jul 02 13:44:02.213645 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:02.213371 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58dfdeb1-5afc-4881-8dd2-3ee2964032c2","Type":"ContainerDied","Data":"3cd66691a3e98e7f52fac71496dffc840a323b8828cb15cb8eb387458295d6fd"} Jul 02 13:44:02.213645 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:02.213461 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58dfdeb1-5afc-4881-8dd2-3ee2964032c2","Type":"ContainerStarted","Data":"82d71341fdc55fb8be76723b946333c5689e01baa38997d67678b07bfde36523"} Jul 02 13:44:02.363168 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:02.363133 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef56f414-16ad-4597-83e3-46b3f1e93c15" path="/var/lib/kubelet/pods/ef56f414-16ad-4597-83e3-46b3f1e93c15/volumes" Jul 02 13:44:03.222860 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:03.222826 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58dfdeb1-5afc-4881-8dd2-3ee2964032c2","Type":"ContainerStarted","Data":"4534d86b859fd1d4ea73a59f2ee70e865a7dde46839497323ad6b7a6edaffe5d"} Jul 02 13:44:03.223065 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:03.222895 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58dfdeb1-5afc-4881-8dd2-3ee2964032c2","Type":"ContainerStarted","Data":"261bd190315961d20670b1f11c24db455ce89eef1025f23248aac6ecc29f2aae"} Jul 02 13:44:03.223065 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:03.222925 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58dfdeb1-5afc-4881-8dd2-3ee2964032c2","Type":"ContainerStarted","Data":"e9ce1743627924b2c31441e95ca969dcd297a2d905b1141705b730e9ce26090a"} Jul 02 13:44:03.223065 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:03.222950 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58dfdeb1-5afc-4881-8dd2-3ee2964032c2","Type":"ContainerStarted","Data":"a00bb93405e7b308a13a8de4a78956495edcbf6ea6456dd7eb490ff0b7b97dae"} Jul 02 13:44:03.223065 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:03.223008 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58dfdeb1-5afc-4881-8dd2-3ee2964032c2","Type":"ContainerStarted","Data":"e03a8645ddf9999f1d6674c8691f2847d680549c4bc4cde46724b5ec26046240"} Jul 02 13:44:03.223065 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:03.223033 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58dfdeb1-5afc-4881-8dd2-3ee2964032c2","Type":"ContainerStarted","Data":"1549574623ab66a6c0a8ed5917548ce5825381c16a306c51953a5a574ceb5fac"} Jul 02 13:44:03.257932 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:03.257867 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.242978042 podStartE2EDuration="2.257844061s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.866019ms" isStatefulPod=false podCreationTimestamp="2026-07-02 13:44:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:03.255938288 +0000 UTC m=+149.382074807" watchObservedRunningTime="2026-07-02 13:44:03.257844061 +0000 UTC m=+149.383980567" Jul 02 13:44:04.289106 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.289070 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-546998c87b-ktm8r"] Jul 02 13:44:04.292079 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.292056 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.295148 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.295125 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 02 13:44:04.295315 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.295161 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 02 13:44:04.295315 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.295212 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 02 13:44:04.296246 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.296228 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-jdtcg\"" Jul 02 13:44:04.296904 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.296887 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 02 13:44:04.297232 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.297206 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 02 13:44:04.303959 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.303929 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 02 13:44:04.304646 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.304624 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-546998c87b-ktm8r"] Jul 02 13:44:04.314575 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.314538 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-secret-telemeter-client\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.314575 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.314577 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-metrics-client-ca\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.314843 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.314627 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnhvj\" (UniqueName: \"kubernetes.io/projected/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-kube-api-access-bnhvj\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.314843 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.314652 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-serving-certs-ca-bundle\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.314985 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.314837 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-telemeter-trusted-ca-bundle\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.315051 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.315006 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-federate-client-tls\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.315051 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.315045 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.315181 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.315095 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-telemeter-client-tls\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.416162 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.416128 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-telemeter-trusted-ca-bundle\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.416377 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.416192 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-federate-client-tls\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.416377 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.416290 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.416377 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.416316 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-telemeter-client-tls\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.416377 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.416366 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-secret-telemeter-client\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.416596 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.416389 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-metrics-client-ca\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.416596 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.416421 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnhvj\" (UniqueName: \"kubernetes.io/projected/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-kube-api-access-bnhvj\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.416596 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.416445 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-serving-certs-ca-bundle\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.417177 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.417153 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-serving-certs-ca-bundle\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.417177 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.417169 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-telemeter-trusted-ca-bundle\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.417685 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.417661 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-metrics-client-ca\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.419414 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.419356 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-federate-client-tls\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.419530 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.419444 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.419596 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.419577 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-telemeter-client-tls\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.420902 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.419871 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-secret-telemeter-client\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.425358 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.425329 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnhvj\" (UniqueName: \"kubernetes.io/projected/17574a8b-b188-4b49-8fb9-8ee93bc54bbf-kube-api-access-bnhvj\") pod \"telemeter-client-546998c87b-ktm8r\" (UID: \"17574a8b-b188-4b49-8fb9-8ee93bc54bbf\") " pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.605259 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.605166 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" Jul 02 13:44:04.615865 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.615828 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 13:44:04.616259 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.616223 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="prometheus" containerID="cri-o://6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df" gracePeriod=600 Jul 02 13:44:04.616517 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.616248 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="kube-rbac-proxy" containerID="cri-o://0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8" gracePeriod=600 Jul 02 13:44:04.616648 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.616248 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="thanos-sidecar" containerID="cri-o://20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466" gracePeriod=600 Jul 02 13:44:04.616837 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.616279 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="kube-rbac-proxy-web" containerID="cri-o://b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c" gracePeriod=600 Jul 02 13:44:04.616837 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.616301 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="config-reloader" containerID="cri-o://67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c" gracePeriod=600 Jul 02 13:44:04.616837 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.616306 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="kube-rbac-proxy-thanos" containerID="cri-o://2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87" gracePeriod=600 Jul 02 13:44:04.785703 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.785614 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-546998c87b-ktm8r"] Jul 02 13:44:04.788489 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:44:04.788453 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17574a8b_b188_4b49_8fb9_8ee93bc54bbf.slice/crio-6402d63e311612602282eb2f8e9daa536db2bbca0d8252a7c5c7a2c0e282cc16 WatchSource:0}: Error finding container 6402d63e311612602282eb2f8e9daa536db2bbca0d8252a7c5c7a2c0e282cc16: Status 404 returned error can't find the container with id 6402d63e311612602282eb2f8e9daa536db2bbca0d8252a7c5c7a2c0e282cc16 Jul 02 13:44:04.887078 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.887045 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:04.921266 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.921226 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-rulefiles-0\") pod \"834585dd-3bd9-4116-af0e-22bb1287a590\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " Jul 02 13:44:04.921434 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.921298 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-rulefiles-1\") pod \"834585dd-3bd9-4116-af0e-22bb1287a590\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " Jul 02 13:44:04.921434 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.921336 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"834585dd-3bd9-4116-af0e-22bb1287a590\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " Jul 02 13:44:04.921434 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.921373 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-metrics-client-certs\") pod \"834585dd-3bd9-4116-af0e-22bb1287a590\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " Jul 02 13:44:04.921434 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.921411 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-web-config\") pod \"834585dd-3bd9-4116-af0e-22bb1287a590\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " Jul 02 13:44:04.921657 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.921450 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-db\") pod \"834585dd-3bd9-4116-af0e-22bb1287a590\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " Jul 02 13:44:04.921657 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.921478 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-trusted-ca-bundle\") pod \"834585dd-3bd9-4116-af0e-22bb1287a590\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " Jul 02 13:44:04.921657 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.921504 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-configmap-kubelet-serving-ca-bundle\") pod \"834585dd-3bd9-4116-af0e-22bb1287a590\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " Jul 02 13:44:04.921657 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.921533 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-grpc-tls\") pod \"834585dd-3bd9-4116-af0e-22bb1287a590\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " Jul 02 13:44:04.921657 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.921569 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-configmap-metrics-client-ca\") pod \"834585dd-3bd9-4116-af0e-22bb1287a590\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " Jul 02 13:44:04.921657 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.921611 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-configmap-serving-certs-ca-bundle\") pod \"834585dd-3bd9-4116-af0e-22bb1287a590\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " Jul 02 13:44:04.921657 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.921637 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/834585dd-3bd9-4116-af0e-22bb1287a590-config-out\") pod \"834585dd-3bd9-4116-af0e-22bb1287a590\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " Jul 02 13:44:04.922002 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.921666 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"834585dd-3bd9-4116-af0e-22bb1287a590\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " Jul 02 13:44:04.922002 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.921697 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/834585dd-3bd9-4116-af0e-22bb1287a590-tls-assets\") pod \"834585dd-3bd9-4116-af0e-22bb1287a590\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " Jul 02 13:44:04.922002 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.921723 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hx6z\" (UniqueName: \"kubernetes.io/projected/834585dd-3bd9-4116-af0e-22bb1287a590-kube-api-access-8hx6z\") pod \"834585dd-3bd9-4116-af0e-22bb1287a590\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " Jul 02 13:44:04.922002 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.921756 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-kube-rbac-proxy\") pod \"834585dd-3bd9-4116-af0e-22bb1287a590\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " Jul 02 13:44:04.922002 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.921786 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-rulefiles-2\") pod \"834585dd-3bd9-4116-af0e-22bb1287a590\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " Jul 02 13:44:04.922002 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.921814 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-prometheus-k8s-tls\") pod \"834585dd-3bd9-4116-af0e-22bb1287a590\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " Jul 02 13:44:04.922002 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.921851 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-config\") pod \"834585dd-3bd9-4116-af0e-22bb1287a590\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " Jul 02 13:44:04.922002 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.921881 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-thanos-prometheus-http-client-file\") pod \"834585dd-3bd9-4116-af0e-22bb1287a590\" (UID: \"834585dd-3bd9-4116-af0e-22bb1287a590\") " Jul 02 13:44:04.922714 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.922681 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "834585dd-3bd9-4116-af0e-22bb1287a590" (UID: "834585dd-3bd9-4116-af0e-22bb1287a590"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:44:04.922926 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.922816 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "834585dd-3bd9-4116-af0e-22bb1287a590" (UID: "834585dd-3bd9-4116-af0e-22bb1287a590"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:44:04.923044 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.923015 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "834585dd-3bd9-4116-af0e-22bb1287a590" (UID: "834585dd-3bd9-4116-af0e-22bb1287a590"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:44:04.924254 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.924012 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "834585dd-3bd9-4116-af0e-22bb1287a590" (UID: "834585dd-3bd9-4116-af0e-22bb1287a590"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 13:44:04.925653 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.924362 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "834585dd-3bd9-4116-af0e-22bb1287a590" (UID: "834585dd-3bd9-4116-af0e-22bb1287a590"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:44:04.925653 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.925041 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/834585dd-3bd9-4116-af0e-22bb1287a590-kube-api-access-8hx6z" (OuterVolumeSpecName: "kube-api-access-8hx6z") pod "834585dd-3bd9-4116-af0e-22bb1287a590" (UID: "834585dd-3bd9-4116-af0e-22bb1287a590"). InnerVolumeSpecName "kube-api-access-8hx6z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:44:04.926613 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.926228 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "834585dd-3bd9-4116-af0e-22bb1287a590" (UID: "834585dd-3bd9-4116-af0e-22bb1287a590"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:44:04.926613 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.926337 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "834585dd-3bd9-4116-af0e-22bb1287a590" (UID: "834585dd-3bd9-4116-af0e-22bb1287a590"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:44:04.927374 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.927184 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "834585dd-3bd9-4116-af0e-22bb1287a590" (UID: "834585dd-3bd9-4116-af0e-22bb1287a590"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:44:04.927635 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.927525 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "834585dd-3bd9-4116-af0e-22bb1287a590" (UID: "834585dd-3bd9-4116-af0e-22bb1287a590"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:44:04.927843 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.927802 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "834585dd-3bd9-4116-af0e-22bb1287a590" (UID: "834585dd-3bd9-4116-af0e-22bb1287a590"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:44:04.928134 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.928103 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "834585dd-3bd9-4116-af0e-22bb1287a590" (UID: "834585dd-3bd9-4116-af0e-22bb1287a590"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:44:04.928481 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.928444 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/834585dd-3bd9-4116-af0e-22bb1287a590-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "834585dd-3bd9-4116-af0e-22bb1287a590" (UID: "834585dd-3bd9-4116-af0e-22bb1287a590"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:44:04.933144 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.933104 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "834585dd-3bd9-4116-af0e-22bb1287a590" (UID: "834585dd-3bd9-4116-af0e-22bb1287a590"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:44:04.941220 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.941147 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-config" (OuterVolumeSpecName: "config") pod "834585dd-3bd9-4116-af0e-22bb1287a590" (UID: "834585dd-3bd9-4116-af0e-22bb1287a590"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:44:04.941469 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.941432 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/834585dd-3bd9-4116-af0e-22bb1287a590-config-out" (OuterVolumeSpecName: "config-out") pod "834585dd-3bd9-4116-af0e-22bb1287a590" (UID: "834585dd-3bd9-4116-af0e-22bb1287a590"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 13:44:04.941706 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.941633 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "834585dd-3bd9-4116-af0e-22bb1287a590" (UID: "834585dd-3bd9-4116-af0e-22bb1287a590"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:44:04.941896 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.941863 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "834585dd-3bd9-4116-af0e-22bb1287a590" (UID: "834585dd-3bd9-4116-af0e-22bb1287a590"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:44:04.953020 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.952140 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "834585dd-3bd9-4116-af0e-22bb1287a590" (UID: "834585dd-3bd9-4116-af0e-22bb1287a590"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:44:04.955394 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:04.955353 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-web-config" (OuterVolumeSpecName: "web-config") pod "834585dd-3bd9-4116-af0e-22bb1287a590" (UID: "834585dd-3bd9-4116-af0e-22bb1287a590"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:44:05.022873 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.022837 2561 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:05.022873 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.022873 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:05.022873 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.022884 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:05.023155 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.022893 2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:05.023155 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.022903 2561 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-metrics-client-certs\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:05.023155 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.022914 2561 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-web-config\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:05.023155 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.022922 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-db\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:05.023155 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.022934 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:05.023155 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.022948 2561 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:05.023155 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.022959 2561 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-grpc-tls\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:05.023155 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.022967 2561 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-configmap-metrics-client-ca\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:05.023155 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.023001 2561 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:05.023155 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.023010 2561 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/834585dd-3bd9-4116-af0e-22bb1287a590-config-out\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:05.023155 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.023023 2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:05.023155 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.023034 2561 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/834585dd-3bd9-4116-af0e-22bb1287a590-tls-assets\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:05.023155 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.023044 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hx6z\" (UniqueName: \"kubernetes.io/projected/834585dd-3bd9-4116-af0e-22bb1287a590-kube-api-access-8hx6z\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:05.023155 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.023053 2561 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-kube-rbac-proxy\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:05.023155 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.023063 2561 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/834585dd-3bd9-4116-af0e-22bb1287a590-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:05.023155 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.023071 2561 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:05.023155 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.023080 2561 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/834585dd-3bd9-4116-af0e-22bb1287a590-config\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:05.239680 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.239641 2561 generic.go:356] "Generic (PLEG): container finished" podID="834585dd-3bd9-4116-af0e-22bb1287a590" containerID="2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87" exitCode=0 Jul 02 13:44:05.239680 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.239674 2561 generic.go:356] "Generic (PLEG): container finished" podID="834585dd-3bd9-4116-af0e-22bb1287a590" containerID="0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8" exitCode=0 Jul 02 13:44:05.239680 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.239682 2561 generic.go:356] "Generic (PLEG): container finished" podID="834585dd-3bd9-4116-af0e-22bb1287a590" containerID="b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c" exitCode=0 Jul 02 13:44:05.239680 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.239689 2561 generic.go:356] "Generic (PLEG): container finished" podID="834585dd-3bd9-4116-af0e-22bb1287a590" containerID="20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466" exitCode=0 Jul 02 13:44:05.240156 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.239695 2561 generic.go:356] "Generic (PLEG): container finished" podID="834585dd-3bd9-4116-af0e-22bb1287a590" containerID="67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c" exitCode=0 Jul 02 13:44:05.240156 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.239701 2561 generic.go:356] "Generic (PLEG): container finished" podID="834585dd-3bd9-4116-af0e-22bb1287a590" containerID="6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df" exitCode=0 Jul 02 13:44:05.240156 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.239728 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.240156 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.239739 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"834585dd-3bd9-4116-af0e-22bb1287a590","Type":"ContainerDied","Data":"2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87"} Jul 02 13:44:05.240156 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.239835 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"834585dd-3bd9-4116-af0e-22bb1287a590","Type":"ContainerDied","Data":"0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8"} Jul 02 13:44:05.240156 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.239881 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"834585dd-3bd9-4116-af0e-22bb1287a590","Type":"ContainerDied","Data":"b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c"} Jul 02 13:44:05.240156 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.239917 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"834585dd-3bd9-4116-af0e-22bb1287a590","Type":"ContainerDied","Data":"20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466"} Jul 02 13:44:05.240156 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.239984 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"834585dd-3bd9-4116-af0e-22bb1287a590","Type":"ContainerDied","Data":"67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c"} Jul 02 13:44:05.240156 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.239918 2561 scope.go:117] "RemoveContainer" containerID="2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87" Jul 02 13:44:05.240156 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.240027 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"834585dd-3bd9-4116-af0e-22bb1287a590","Type":"ContainerDied","Data":"6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df"} Jul 02 13:44:05.240156 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.240070 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"834585dd-3bd9-4116-af0e-22bb1287a590","Type":"ContainerDied","Data":"ebc605d9e6d84e7fa7d9201cbe15e27a3616dd9e972fe2264ad59b76c8aa91fb"} Jul 02 13:44:05.241469 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.241411 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" event={"ID":"17574a8b-b188-4b49-8fb9-8ee93bc54bbf","Type":"ContainerStarted","Data":"6402d63e311612602282eb2f8e9daa536db2bbca0d8252a7c5c7a2c0e282cc16"} Jul 02 13:44:05.250274 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.250255 2561 scope.go:117] "RemoveContainer" containerID="0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8" Jul 02 13:44:05.259741 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.259717 2561 scope.go:117] "RemoveContainer" containerID="b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c" Jul 02 13:44:05.269172 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.269140 2561 scope.go:117] "RemoveContainer" containerID="20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466" Jul 02 13:44:05.269623 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.269602 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 13:44:05.275351 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.275327 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 13:44:05.279457 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.279405 2561 scope.go:117] "RemoveContainer" containerID="67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c" Jul 02 13:44:05.289064 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.289040 2561 scope.go:117] "RemoveContainer" containerID="6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df" Jul 02 13:44:05.300123 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.300089 2561 scope.go:117] "RemoveContainer" containerID="d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566" Jul 02 13:44:05.304198 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.304171 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 13:44:05.304649 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.304631 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="init-config-reloader" Jul 02 13:44:05.304649 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.304651 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="init-config-reloader" Jul 02 13:44:05.304809 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.304664 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="config-reloader" Jul 02 13:44:05.304809 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.304673 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="config-reloader" Jul 02 13:44:05.304809 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.304681 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="thanos-sidecar" Jul 02 13:44:05.304809 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.304689 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="thanos-sidecar" Jul 02 13:44:05.304809 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.304710 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="kube-rbac-proxy" Jul 02 13:44:05.304809 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.304720 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="kube-rbac-proxy" Jul 02 13:44:05.304809 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.304727 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="kube-rbac-proxy-thanos" Jul 02 13:44:05.304809 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.304735 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="kube-rbac-proxy-thanos" Jul 02 13:44:05.304809 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.304754 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="kube-rbac-proxy-web" Jul 02 13:44:05.304809 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.304761 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="kube-rbac-proxy-web" Jul 02 13:44:05.304809 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.304778 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="prometheus" Jul 02 13:44:05.304809 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.304787 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="prometheus" Jul 02 13:44:05.305517 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.304864 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="kube-rbac-proxy-web" Jul 02 13:44:05.305517 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.304877 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="kube-rbac-proxy-thanos" Jul 02 13:44:05.305517 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.304889 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="config-reloader" Jul 02 13:44:05.305517 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.304899 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="thanos-sidecar" Jul 02 13:44:05.305517 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.304932 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="prometheus" Jul 02 13:44:05.305517 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.304944 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" containerName="kube-rbac-proxy" Jul 02 13:44:05.309242 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.309178 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.309308 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.309276 2561 scope.go:117] "RemoveContainer" containerID="2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87" Jul 02 13:44:05.310456 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:44:05.310356 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87\": container with ID starting with 2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87 not found: ID does not exist" containerID="2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87" Jul 02 13:44:05.310456 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.310397 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87"} err="failed to get container status \"2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87\": rpc error: code = NotFound desc = could not find container \"2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87\": container with ID starting with 2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87 not found: ID does not exist" Jul 02 13:44:05.310456 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.310426 2561 scope.go:117] "RemoveContainer" containerID="0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8" Jul 02 13:44:05.311314 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:44:05.311243 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8\": container with ID starting with 0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8 not found: ID does not exist" containerID="0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8" Jul 02 13:44:05.311314 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.311287 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8"} err="failed to get container status \"0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8\": rpc error: code = NotFound desc = could not find container \"0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8\": container with ID starting with 0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8 not found: ID does not exist" Jul 02 13:44:05.311484 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.311318 2561 scope.go:117] "RemoveContainer" containerID="b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c" Jul 02 13:44:05.311608 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:44:05.311586 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c\": container with ID starting with b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c not found: ID does not exist" containerID="b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c" Jul 02 13:44:05.311667 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.311617 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c"} err="failed to get container status \"b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c\": rpc error: code = NotFound desc = could not find container \"b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c\": container with ID starting with b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c not found: ID does not exist" Jul 02 13:44:05.311667 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.311641 2561 scope.go:117] "RemoveContainer" containerID="20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466" Jul 02 13:44:05.311811 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.311789 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 02 13:44:05.311912 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:44:05.311874 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466\": container with ID starting with 20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466 not found: ID does not exist" containerID="20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466" Jul 02 13:44:05.311912 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.311900 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466"} err="failed to get container status \"20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466\": rpc error: code = NotFound desc = could not find container \"20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466\": container with ID starting with 20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466 not found: ID does not exist" Jul 02 13:44:05.312064 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.311922 2561 scope.go:117] "RemoveContainer" containerID="67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c" Jul 02 13:44:05.312119 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.312072 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4mfd3ag069ovf\"" Jul 02 13:44:05.312119 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.312094 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 02 13:44:05.312241 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.312072 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 02 13:44:05.312330 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.312289 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 02 13:44:05.312330 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.312307 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 02 13:44:05.312693 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:44:05.312626 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c\": container with ID starting with 67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c not found: ID does not exist" containerID="67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c" Jul 02 13:44:05.312693 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.312662 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 02 13:44:05.312693 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.312656 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c"} err="failed to get container status \"67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c\": rpc error: code = NotFound desc = could not find container \"67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c\": container with ID starting with 67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c not found: ID does not exist" Jul 02 13:44:05.312693 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.312686 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-2jnnl\"" Jul 02 13:44:05.312693 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.312692 2561 scope.go:117] "RemoveContainer" containerID="6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df" Jul 02 13:44:05.313118 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.312740 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 02 13:44:05.313118 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.312797 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 02 13:44:05.313118 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.312956 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 02 13:44:05.313118 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.312963 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 02 13:44:05.313333 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:44:05.313183 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df\": container with ID starting with 6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df not found: ID does not exist" containerID="6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df" Jul 02 13:44:05.313333 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.313215 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df"} err="failed to get container status \"6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df\": rpc error: code = NotFound desc = could not find container \"6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df\": container with ID starting with 6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df not found: ID does not exist" Jul 02 13:44:05.313333 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.313235 2561 scope.go:117] "RemoveContainer" containerID="d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566" Jul 02 13:44:05.313333 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.313260 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 02 13:44:05.313588 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:44:05.313529 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566\": container with ID starting with d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566 not found: ID does not exist" containerID="d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566" Jul 02 13:44:05.313588 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.313571 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566"} err="failed to get container status \"d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566\": rpc error: code = NotFound desc = could not find container \"d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566\": container with ID starting with d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566 not found: ID does not exist" Jul 02 13:44:05.313690 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.313592 2561 scope.go:117] "RemoveContainer" containerID="2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87" Jul 02 13:44:05.313871 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.313846 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87"} err="failed to get container status \"2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87\": rpc error: code = NotFound desc = could not find container \"2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87\": container with ID starting with 2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87 not found: ID does not exist" Jul 02 13:44:05.313871 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.313871 2561 scope.go:117] "RemoveContainer" containerID="0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8" Jul 02 13:44:05.314256 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.314157 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8"} err="failed to get container status \"0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8\": rpc error: code = NotFound desc = could not find container \"0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8\": container with ID starting with 0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8 not found: ID does not exist" Jul 02 13:44:05.314256 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.314181 2561 scope.go:117] "RemoveContainer" containerID="b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c" Jul 02 13:44:05.314535 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.314506 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c"} err="failed to get container status \"b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c\": rpc error: code = NotFound desc = could not find container \"b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c\": container with ID starting with b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c not found: ID does not exist" Jul 02 13:44:05.314613 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.314536 2561 scope.go:117] "RemoveContainer" containerID="20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466" Jul 02 13:44:05.314809 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.314782 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466"} err="failed to get container status \"20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466\": rpc error: code = NotFound desc = could not find container \"20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466\": container with ID starting with 20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466 not found: ID does not exist" Jul 02 13:44:05.314873 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.314812 2561 scope.go:117] "RemoveContainer" containerID="67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c" Jul 02 13:44:05.315237 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.315199 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c"} err="failed to get container status \"67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c\": rpc error: code = NotFound desc = could not find container \"67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c\": container with ID starting with 67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c not found: ID does not exist" Jul 02 13:44:05.315237 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.315226 2561 scope.go:117] "RemoveContainer" containerID="6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df" Jul 02 13:44:05.315621 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.315560 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df"} err="failed to get container status \"6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df\": rpc error: code = NotFound desc = could not find container \"6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df\": container with ID starting with 6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df not found: ID does not exist" Jul 02 13:44:05.315621 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.315608 2561 scope.go:117] "RemoveContainer" containerID="d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566" Jul 02 13:44:05.315881 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.315850 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566"} err="failed to get container status \"d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566\": rpc error: code = NotFound desc = could not find container \"d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566\": container with ID starting with d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566 not found: ID does not exist" Jul 02 13:44:05.315986 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.315883 2561 scope.go:117] "RemoveContainer" containerID="2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87" Jul 02 13:44:05.316263 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.316240 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87"} err="failed to get container status \"2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87\": rpc error: code = NotFound desc = could not find container \"2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87\": container with ID starting with 2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87 not found: ID does not exist" Jul 02 13:44:05.316374 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.316362 2561 scope.go:117] "RemoveContainer" containerID="0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8" Jul 02 13:44:05.317020 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.316874 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 02 13:44:05.317020 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.316946 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8"} err="failed to get container status \"0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8\": rpc error: code = NotFound desc = could not find container \"0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8\": container with ID starting with 0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8 not found: ID does not exist" Jul 02 13:44:05.317020 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.316988 2561 scope.go:117] "RemoveContainer" containerID="b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c" Jul 02 13:44:05.317286 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.317257 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c"} err="failed to get container status \"b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c\": rpc error: code = NotFound desc = could not find container \"b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c\": container with ID starting with b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c not found: ID does not exist" Jul 02 13:44:05.317354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.317288 2561 scope.go:117] "RemoveContainer" containerID="20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466" Jul 02 13:44:05.317566 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.317547 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 02 13:44:05.317656 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.317626 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466"} err="failed to get container status \"20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466\": rpc error: code = NotFound desc = could not find container \"20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466\": container with ID starting with 20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466 not found: ID does not exist" Jul 02 13:44:05.317725 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.317660 2561 scope.go:117] "RemoveContainer" containerID="67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c" Jul 02 13:44:05.318375 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.318351 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c"} err="failed to get container status \"67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c\": rpc error: code = NotFound desc = could not find container \"67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c\": container with ID starting with 67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c not found: ID does not exist" Jul 02 13:44:05.318472 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.318377 2561 scope.go:117] "RemoveContainer" containerID="6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df" Jul 02 13:44:05.319000 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.318810 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df"} err="failed to get container status \"6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df\": rpc error: code = NotFound desc = could not find container \"6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df\": container with ID starting with 6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df not found: ID does not exist" Jul 02 13:44:05.319000 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.318842 2561 scope.go:117] "RemoveContainer" containerID="d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566" Jul 02 13:44:05.319684 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.319170 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566"} err="failed to get container status \"d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566\": rpc error: code = NotFound desc = could not find container \"d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566\": container with ID starting with d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566 not found: ID does not exist" Jul 02 13:44:05.319684 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.319225 2561 scope.go:117] "RemoveContainer" containerID="2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87" Jul 02 13:44:05.319684 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.319471 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87"} err="failed to get container status \"2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87\": rpc error: code = NotFound desc = could not find container \"2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87\": container with ID starting with 2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87 not found: ID does not exist" Jul 02 13:44:05.319684 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.319492 2561 scope.go:117] "RemoveContainer" containerID="0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8" Jul 02 13:44:05.319955 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.319709 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8"} err="failed to get container status \"0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8\": rpc error: code = NotFound desc = could not find container \"0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8\": container with ID starting with 0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8 not found: ID does not exist" Jul 02 13:44:05.319955 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.319731 2561 scope.go:117] "RemoveContainer" containerID="b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c" Jul 02 13:44:05.320153 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.320029 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c"} err="failed to get container status \"b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c\": rpc error: code = NotFound desc = could not find container \"b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c\": container with ID starting with b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c not found: ID does not exist" Jul 02 13:44:05.320153 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.320053 2561 scope.go:117] "RemoveContainer" containerID="20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466" Jul 02 13:44:05.320394 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.320363 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466"} err="failed to get container status \"20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466\": rpc error: code = NotFound desc = could not find container \"20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466\": container with ID starting with 20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466 not found: ID does not exist" Jul 02 13:44:05.320394 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.320389 2561 scope.go:117] "RemoveContainer" containerID="67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c" Jul 02 13:44:05.320687 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.320663 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c"} err="failed to get container status \"67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c\": rpc error: code = NotFound desc = could not find container \"67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c\": container with ID starting with 67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c not found: ID does not exist" Jul 02 13:44:05.320811 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.320689 2561 scope.go:117] "RemoveContainer" containerID="6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df" Jul 02 13:44:05.321108 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.321042 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 02 13:44:05.321108 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.321057 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df"} err="failed to get container status \"6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df\": rpc error: code = NotFound desc = could not find container \"6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df\": container with ID starting with 6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df not found: ID does not exist" Jul 02 13:44:05.321108 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.321079 2561 scope.go:117] "RemoveContainer" containerID="d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566" Jul 02 13:44:05.321373 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.321354 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566"} err="failed to get container status \"d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566\": rpc error: code = NotFound desc = could not find container \"d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566\": container with ID starting with d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566 not found: ID does not exist" Jul 02 13:44:05.321459 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.321375 2561 scope.go:117] "RemoveContainer" containerID="2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87" Jul 02 13:44:05.321633 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.321607 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87"} err="failed to get container status \"2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87\": rpc error: code = NotFound desc = could not find container \"2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87\": container with ID starting with 2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87 not found: ID does not exist" Jul 02 13:44:05.321694 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.321635 2561 scope.go:117] "RemoveContainer" containerID="0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8" Jul 02 13:44:05.321932 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.321910 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8"} err="failed to get container status \"0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8\": rpc error: code = NotFound desc = could not find container \"0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8\": container with ID starting with 0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8 not found: ID does not exist" Jul 02 13:44:05.322015 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.321935 2561 scope.go:117] "RemoveContainer" containerID="b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c" Jul 02 13:44:05.322202 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.322182 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c"} err="failed to get container status \"b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c\": rpc error: code = NotFound desc = could not find container \"b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c\": container with ID starting with b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c not found: ID does not exist" Jul 02 13:44:05.322202 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.322202 2561 scope.go:117] "RemoveContainer" containerID="20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466" Jul 02 13:44:05.322444 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.322410 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466"} err="failed to get container status \"20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466\": rpc error: code = NotFound desc = could not find container \"20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466\": container with ID starting with 20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466 not found: ID does not exist" Jul 02 13:44:05.322444 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.322434 2561 scope.go:117] "RemoveContainer" containerID="67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c" Jul 02 13:44:05.322676 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.322654 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c"} err="failed to get container status \"67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c\": rpc error: code = NotFound desc = could not find container \"67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c\": container with ID starting with 67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c not found: ID does not exist" Jul 02 13:44:05.322744 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.322678 2561 scope.go:117] "RemoveContainer" containerID="6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df" Jul 02 13:44:05.323048 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.323015 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df"} err="failed to get container status \"6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df\": rpc error: code = NotFound desc = could not find container \"6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df\": container with ID starting with 6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df not found: ID does not exist" Jul 02 13:44:05.323048 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.323047 2561 scope.go:117] "RemoveContainer" containerID="d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566" Jul 02 13:44:05.323278 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.323258 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566"} err="failed to get container status \"d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566\": rpc error: code = NotFound desc = could not find container \"d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566\": container with ID starting with d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566 not found: ID does not exist" Jul 02 13:44:05.323359 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.323280 2561 scope.go:117] "RemoveContainer" containerID="2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87" Jul 02 13:44:05.323542 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.323518 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87"} err="failed to get container status \"2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87\": rpc error: code = NotFound desc = could not find container \"2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87\": container with ID starting with 2eec4adba68558dbb33f66a60815b2a1a83a9ba48e74ea4f21ac53d3da09ed87 not found: ID does not exist" Jul 02 13:44:05.323542 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.323543 2561 scope.go:117] "RemoveContainer" containerID="0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8" Jul 02 13:44:05.323787 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.323770 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8"} err="failed to get container status \"0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8\": rpc error: code = NotFound desc = could not find container \"0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8\": container with ID starting with 0fc2d99d6762ac7b1b2d467baefb9f6aef0f87d24c733e1494ac6b20730806b8 not found: ID does not exist" Jul 02 13:44:05.323833 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.323788 2561 scope.go:117] "RemoveContainer" containerID="b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c" Jul 02 13:44:05.324133 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.324106 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c"} err="failed to get container status \"b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c\": rpc error: code = NotFound desc = could not find container \"b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c\": container with ID starting with b2462da0e8648817fb49e1be20afee63e984bdcd5ee5919623340ef6536fef9c not found: ID does not exist" Jul 02 13:44:05.324222 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.324134 2561 scope.go:117] "RemoveContainer" containerID="20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466" Jul 02 13:44:05.324362 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.324345 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466"} err="failed to get container status \"20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466\": rpc error: code = NotFound desc = could not find container \"20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466\": container with ID starting with 20e6ec670dab8ea3061be1242a513cc59bcbb10c87e17e5be404d9d3a17f8466 not found: ID does not exist" Jul 02 13:44:05.324408 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.324363 2561 scope.go:117] "RemoveContainer" containerID="67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c" Jul 02 13:44:05.325493 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.324592 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c"} err="failed to get container status \"67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c\": rpc error: code = NotFound desc = could not find container \"67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c\": container with ID starting with 67482ac017969a5cf1e390675ec3a08c46cae49e6cd80e8169448c0bea88db3c not found: ID does not exist" Jul 02 13:44:05.325493 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.324616 2561 scope.go:117] "RemoveContainer" containerID="6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df" Jul 02 13:44:05.325493 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.324892 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df"} err="failed to get container status \"6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df\": rpc error: code = NotFound desc = could not find container \"6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df\": container with ID starting with 6e74961a7909ae3e687d45693d54361ebd9638753fc1efc39b0168f78282e2df not found: ID does not exist" Jul 02 13:44:05.325493 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.324909 2561 scope.go:117] "RemoveContainer" containerID="d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566" Jul 02 13:44:05.325813 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.325786 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566"} err="failed to get container status \"d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566\": rpc error: code = NotFound desc = could not find container \"d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566\": container with ID starting with d437f48ef1479d452f554481f00a86607f0b651562c846ab677244a72342b566 not found: ID does not exist" Jul 02 13:44:05.329552 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.329472 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 13:44:05.428465 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.428392 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.428596 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.428486 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b11f3c5-7837-46ca-8ddf-bb25be65d879-config-out\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.428596 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.428514 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.428596 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.428543 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8b11f3c5-7837-46ca-8ddf-bb25be65d879-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.428596 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.428578 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b11f3c5-7837-46ca-8ddf-bb25be65d879-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.428820 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.428606 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b11f3c5-7837-46ca-8ddf-bb25be65d879-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.428820 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.428632 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.428820 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.428667 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.428820 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.428702 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b11f3c5-7837-46ca-8ddf-bb25be65d879-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.428820 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.428741 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8b11f3c5-7837-46ca-8ddf-bb25be65d879-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.428820 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.428785 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.428820 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.428809 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b11f3c5-7837-46ca-8ddf-bb25be65d879-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.429130 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.428833 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-web-config\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.429130 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.428860 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lfxd\" (UniqueName: \"kubernetes.io/projected/8b11f3c5-7837-46ca-8ddf-bb25be65d879-kube-api-access-9lfxd\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.429130 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.428924 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-config\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.429130 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.428949 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8b11f3c5-7837-46ca-8ddf-bb25be65d879-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.429130 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.429004 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8b11f3c5-7837-46ca-8ddf-bb25be65d879-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.429130 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.429042 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b11f3c5-7837-46ca-8ddf-bb25be65d879-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.429130 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.429064 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.429130 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.429102 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.529930 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.529844 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.529930 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.529908 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.530185 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.529942 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b11f3c5-7837-46ca-8ddf-bb25be65d879-config-out\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.530185 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.529966 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.530261 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.530185 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8b11f3c5-7837-46ca-8ddf-bb25be65d879-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.530261 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.530240 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b11f3c5-7837-46ca-8ddf-bb25be65d879-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.530343 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.530277 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b11f3c5-7837-46ca-8ddf-bb25be65d879-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.530343 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.530310 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.530440 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.530352 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.530440 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.530379 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b11f3c5-7837-46ca-8ddf-bb25be65d879-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.530440 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.530416 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8b11f3c5-7837-46ca-8ddf-bb25be65d879-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.530591 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.530461 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.530591 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.530487 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b11f3c5-7837-46ca-8ddf-bb25be65d879-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.530591 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.530513 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-web-config\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.530591 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.530538 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lfxd\" (UniqueName: \"kubernetes.io/projected/8b11f3c5-7837-46ca-8ddf-bb25be65d879-kube-api-access-9lfxd\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.530787 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.530609 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-config\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.530787 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.530632 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8b11f3c5-7837-46ca-8ddf-bb25be65d879-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.530787 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.530666 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8b11f3c5-7837-46ca-8ddf-bb25be65d879-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.530787 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.530703 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b11f3c5-7837-46ca-8ddf-bb25be65d879-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.530787 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.530726 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.531058 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.530831 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8b11f3c5-7837-46ca-8ddf-bb25be65d879-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.531347 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.531310 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b11f3c5-7837-46ca-8ddf-bb25be65d879-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.531519 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.531493 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b11f3c5-7837-46ca-8ddf-bb25be65d879-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.531919 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.531894 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8b11f3c5-7837-46ca-8ddf-bb25be65d879-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.532072 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.532049 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8b11f3c5-7837-46ca-8ddf-bb25be65d879-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.533411 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.533099 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.533411 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.533166 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8b11f3c5-7837-46ca-8ddf-bb25be65d879-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.533562 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.533450 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.533622 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.533572 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8b11f3c5-7837-46ca-8ddf-bb25be65d879-config-out\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.535093 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.534750 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.535093 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.534935 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.535256 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.535232 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8b11f3c5-7837-46ca-8ddf-bb25be65d879-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.535821 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.535799 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b11f3c5-7837-46ca-8ddf-bb25be65d879-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.536220 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.536182 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.536738 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.536323 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.536738 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.536699 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-web-config\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.537135 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.537090 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8b11f3c5-7837-46ca-8ddf-bb25be65d879-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.537629 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.537589 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.537991 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.537947 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8b11f3c5-7837-46ca-8ddf-bb25be65d879-config\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.543307 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.543265 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lfxd\" (UniqueName: \"kubernetes.io/projected/8b11f3c5-7837-46ca-8ddf-bb25be65d879-kube-api-access-9lfxd\") pod \"prometheus-k8s-0\" (UID: \"8b11f3c5-7837-46ca-8ddf-bb25be65d879\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.621691 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.621653 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:05.782692 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:05.782662 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 13:44:06.189317 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:44:06.189253 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b11f3c5_7837_46ca_8ddf_bb25be65d879.slice/crio-100579b78f44618eff2c2c2589f833e3dbcfcba68e5793df21f2f63a6acd5194 WatchSource:0}: Error finding container 100579b78f44618eff2c2c2589f833e3dbcfcba68e5793df21f2f63a6acd5194: Status 404 returned error can't find the container with id 100579b78f44618eff2c2c2589f833e3dbcfcba68e5793df21f2f63a6acd5194 Jul 02 13:44:06.249379 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:06.249348 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b11f3c5-7837-46ca-8ddf-bb25be65d879","Type":"ContainerStarted","Data":"100579b78f44618eff2c2c2589f833e3dbcfcba68e5793df21f2f63a6acd5194"} Jul 02 13:44:06.362351 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:06.362322 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="834585dd-3bd9-4116-af0e-22bb1287a590" path="/var/lib/kubelet/pods/834585dd-3bd9-4116-af0e-22bb1287a590/volumes" Jul 02 13:44:07.254962 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:07.254927 2561 generic.go:356] "Generic (PLEG): container finished" podID="8b11f3c5-7837-46ca-8ddf-bb25be65d879" containerID="c07d50341d4d00349e7dec87fc80f1c9bb8056ab244e3a4bce409e0484c0ffdc" exitCode=0 Jul 02 13:44:07.255164 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:07.255025 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b11f3c5-7837-46ca-8ddf-bb25be65d879","Type":"ContainerDied","Data":"c07d50341d4d00349e7dec87fc80f1c9bb8056ab244e3a4bce409e0484c0ffdc"} Jul 02 13:44:07.257081 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:07.257058 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" event={"ID":"17574a8b-b188-4b49-8fb9-8ee93bc54bbf","Type":"ContainerStarted","Data":"c38bb1ddde9841b360a52082552135b6f24d6958c60ca1ccc672a790623aaedf"} Jul 02 13:44:07.257184 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:07.257102 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" event={"ID":"17574a8b-b188-4b49-8fb9-8ee93bc54bbf","Type":"ContainerStarted","Data":"a629d727417ee248eca4d39e883e9d588c414d54ea8074536d4e4e46be890e66"} Jul 02 13:44:07.257184 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:07.257131 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" event={"ID":"17574a8b-b188-4b49-8fb9-8ee93bc54bbf","Type":"ContainerStarted","Data":"76071569b82478e55cbfb1b6d61956275f595823c86c445013879d82e751ee27"} Jul 02 13:44:07.354812 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:07.354740 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-546998c87b-ktm8r" podStartSLOduration=1.9264437129999998 podStartE2EDuration="3.354719443s" totalImagesPullingTime="1.42827573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:44:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:07.354267358 +0000 UTC m=+153.480403864" watchObservedRunningTime="2026-07-02 13:44:07.354719443 +0000 UTC m=+153.480855954" Jul 02 13:44:08.203810 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.203777 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8994dd95c-drhh8"] Jul 02 13:44:08.206239 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.206218 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.221531 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.221505 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8994dd95c-drhh8"] Jul 02 13:44:08.258787 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.258750 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-service-ca\") pod \"console-8994dd95c-drhh8\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.258936 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.258807 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-trusted-ca-bundle\") pod \"console-8994dd95c-drhh8\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.258936 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.258872 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-console-config\") pod \"console-8994dd95c-drhh8\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.258936 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.258904 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-oauth-serving-cert\") pod \"console-8994dd95c-drhh8\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.259086 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.259018 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-console-oauth-config\") pod \"console-8994dd95c-drhh8\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.259086 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.259039 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9d4x\" (UniqueName: \"kubernetes.io/projected/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-kube-api-access-n9d4x\") pod \"console-8994dd95c-drhh8\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.259086 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.259068 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-console-serving-cert\") pod \"console-8994dd95c-drhh8\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.265757 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.265723 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b11f3c5-7837-46ca-8ddf-bb25be65d879","Type":"ContainerStarted","Data":"a6dd4b134a42b12ec91c3d970d7d82ac2a2cebdbe8e3f93a7fc992302353ef43"} Jul 02 13:44:08.265888 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.265805 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b11f3c5-7837-46ca-8ddf-bb25be65d879","Type":"ContainerStarted","Data":"a6af8849bbee05ea800bbdd85916f441dbe4b75258535d8976fe25166a0f0ef2"} Jul 02 13:44:08.265888 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.265837 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b11f3c5-7837-46ca-8ddf-bb25be65d879","Type":"ContainerStarted","Data":"5e71f8df98ebf64400602a8fd93f6135ba6df93b3c101d2a9c5d17abcfb082f6"} Jul 02 13:44:08.265888 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.265863 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b11f3c5-7837-46ca-8ddf-bb25be65d879","Type":"ContainerStarted","Data":"33e08a6bad8a691f0af7fdeecd9700fbe68f2da16bdf462ea28d3351a5681b07"} Jul 02 13:44:08.265888 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.265887 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b11f3c5-7837-46ca-8ddf-bb25be65d879","Type":"ContainerStarted","Data":"d4c88648a59e5dbd163bb75ea85b64a0dd17b50d495903a6f5a79feda9aea946"} Jul 02 13:44:08.266045 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.265918 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8b11f3c5-7837-46ca-8ddf-bb25be65d879","Type":"ContainerStarted","Data":"68a2500a9d586a76457d042e93975e3b07f3b32da3bfe22a46387c23b03b0624"} Jul 02 13:44:08.295637 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.295573 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.268583516 podStartE2EDuration="3.295549464s" totalImagesPullingTime="0s" totalInitContainerRuntime="26.965948ms" isStatefulPod=false podCreationTimestamp="2026-07-02 13:44:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:08.294525839 +0000 UTC m=+154.420662354" watchObservedRunningTime="2026-07-02 13:44:08.295549464 +0000 UTC m=+154.421685969" Jul 02 13:44:08.359718 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.359686 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-console-serving-cert\") pod \"console-8994dd95c-drhh8\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.359923 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.359800 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-service-ca\") pod \"console-8994dd95c-drhh8\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.360182 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.360154 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-trusted-ca-bundle\") pod \"console-8994dd95c-drhh8\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.360443 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.360419 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-console-config\") pod \"console-8994dd95c-drhh8\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.360530 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.360490 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-oauth-serving-cert\") pod \"console-8994dd95c-drhh8\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.360600 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.360536 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-service-ca\") pod \"console-8994dd95c-drhh8\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.361046 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.361020 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-console-oauth-config\") pod \"console-8994dd95c-drhh8\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.361130 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.361064 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9d4x\" (UniqueName: \"kubernetes.io/projected/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-kube-api-access-n9d4x\") pod \"console-8994dd95c-drhh8\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.361394 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.361370 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-console-config\") pod \"console-8994dd95c-drhh8\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.361588 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.361568 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-trusted-ca-bundle\") pod \"console-8994dd95c-drhh8\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.362133 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.361954 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-oauth-serving-cert\") pod \"console-8994dd95c-drhh8\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.362778 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.362752 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-console-serving-cert\") pod \"console-8994dd95c-drhh8\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.367678 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.367652 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-console-oauth-config\") pod \"console-8994dd95c-drhh8\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.370389 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.370357 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9d4x\" (UniqueName: \"kubernetes.io/projected/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-kube-api-access-n9d4x\") pod \"console-8994dd95c-drhh8\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.514959 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.514914 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:08.660205 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:08.660179 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8994dd95c-drhh8"] Jul 02 13:44:08.662526 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:44:08.662494 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1be8b6d_4dc5_4190_a2ce_142743bb3b71.slice/crio-2c315d783c549809e3d2cc33472b7a076efae962596c92d2a4c0d6716ab98ca5 WatchSource:0}: Error finding container 2c315d783c549809e3d2cc33472b7a076efae962596c92d2a4c0d6716ab98ca5: Status 404 returned error can't find the container with id 2c315d783c549809e3d2cc33472b7a076efae962596c92d2a4c0d6716ab98ca5 Jul 02 13:44:09.271904 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:09.271863 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8994dd95c-drhh8" event={"ID":"e1be8b6d-4dc5-4190-a2ce-142743bb3b71","Type":"ContainerStarted","Data":"0cc500981f7739631440011244a7811af4837060a5046755df92e44caa487c32"} Jul 02 13:44:09.272306 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:09.271923 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8994dd95c-drhh8" event={"ID":"e1be8b6d-4dc5-4190-a2ce-142743bb3b71","Type":"ContainerStarted","Data":"2c315d783c549809e3d2cc33472b7a076efae962596c92d2a4c0d6716ab98ca5"} Jul 02 13:44:09.293665 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:09.293613 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8994dd95c-drhh8" podStartSLOduration=1.2935971 podStartE2EDuration="1.2935971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:44:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:44:09.290929265 +0000 UTC m=+155.417065770" watchObservedRunningTime="2026-07-02 13:44:09.2935971 +0000 UTC m=+155.419733605" Jul 02 13:44:10.621807 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:10.621768 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:44:18.516148 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:18.516109 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:18.516574 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:18.516173 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:18.521046 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:18.521021 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:19.322857 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:19.322830 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:44:19.374831 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:19.374796 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b79bbc56f-sm94p"] Jul 02 13:44:44.396101 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.396052 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7b79bbc56f-sm94p" podUID="fb83e970-4eb8-4518-bec4-e7127c3f21df" containerName="console" containerID="cri-o://4d7616c3099c7e665e48f78e8e1e56deaeb619e2a3331d2acc88464299c0c73e" gracePeriod=15 Jul 02 13:44:44.642717 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.642689 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:44:44.690566 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.690482 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-trusted-ca-bundle\") pod \"fb83e970-4eb8-4518-bec4-e7127c3f21df\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " Jul 02 13:44:44.690566 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.690524 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-service-ca\") pod \"fb83e970-4eb8-4518-bec4-e7127c3f21df\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " Jul 02 13:44:44.690566 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.690557 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-oauth-serving-cert\") pod \"fb83e970-4eb8-4518-bec4-e7127c3f21df\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " Jul 02 13:44:44.690820 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.690591 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmhfq\" (UniqueName: \"kubernetes.io/projected/fb83e970-4eb8-4518-bec4-e7127c3f21df-kube-api-access-tmhfq\") pod \"fb83e970-4eb8-4518-bec4-e7127c3f21df\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " Jul 02 13:44:44.690820 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.690632 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-console-config\") pod \"fb83e970-4eb8-4518-bec4-e7127c3f21df\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " Jul 02 13:44:44.690820 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.690658 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb83e970-4eb8-4518-bec4-e7127c3f21df-console-serving-cert\") pod \"fb83e970-4eb8-4518-bec4-e7127c3f21df\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " Jul 02 13:44:44.690820 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.690684 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb83e970-4eb8-4518-bec4-e7127c3f21df-console-oauth-config\") pod \"fb83e970-4eb8-4518-bec4-e7127c3f21df\" (UID: \"fb83e970-4eb8-4518-bec4-e7127c3f21df\") " Jul 02 13:44:44.691037 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.691000 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-service-ca" (OuterVolumeSpecName: "service-ca") pod "fb83e970-4eb8-4518-bec4-e7127c3f21df" (UID: "fb83e970-4eb8-4518-bec4-e7127c3f21df"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:44:44.691037 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.691015 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fb83e970-4eb8-4518-bec4-e7127c3f21df" (UID: "fb83e970-4eb8-4518-bec4-e7127c3f21df"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:44:44.691142 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.691033 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-console-config" (OuterVolumeSpecName: "console-config") pod "fb83e970-4eb8-4518-bec4-e7127c3f21df" (UID: "fb83e970-4eb8-4518-bec4-e7127c3f21df"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:44:44.691142 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.691047 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fb83e970-4eb8-4518-bec4-e7127c3f21df" (UID: "fb83e970-4eb8-4518-bec4-e7127c3f21df"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:44:44.692814 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.692787 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb83e970-4eb8-4518-bec4-e7127c3f21df-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fb83e970-4eb8-4518-bec4-e7127c3f21df" (UID: "fb83e970-4eb8-4518-bec4-e7127c3f21df"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:44:44.692814 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.692807 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb83e970-4eb8-4518-bec4-e7127c3f21df-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fb83e970-4eb8-4518-bec4-e7127c3f21df" (UID: "fb83e970-4eb8-4518-bec4-e7127c3f21df"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:44:44.692952 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.692887 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb83e970-4eb8-4518-bec4-e7127c3f21df-kube-api-access-tmhfq" (OuterVolumeSpecName: "kube-api-access-tmhfq") pod "fb83e970-4eb8-4518-bec4-e7127c3f21df" (UID: "fb83e970-4eb8-4518-bec4-e7127c3f21df"). InnerVolumeSpecName "kube-api-access-tmhfq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:44:44.791669 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.791631 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-trusted-ca-bundle\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:44.791669 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.791662 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-service-ca\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:44.791669 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.791674 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-oauth-serving-cert\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:44.791907 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.791683 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmhfq\" (UniqueName: \"kubernetes.io/projected/fb83e970-4eb8-4518-bec4-e7127c3f21df-kube-api-access-tmhfq\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:44.791907 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.791692 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb83e970-4eb8-4518-bec4-e7127c3f21df-console-config\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:44.791907 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.791700 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb83e970-4eb8-4518-bec4-e7127c3f21df-console-serving-cert\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:44.791907 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:44.791710 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb83e970-4eb8-4518-bec4-e7127c3f21df-console-oauth-config\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:44:45.429534 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:45.429499 2561 generic.go:356] "Generic (PLEG): container finished" podID="fb83e970-4eb8-4518-bec4-e7127c3f21df" containerID="4d7616c3099c7e665e48f78e8e1e56deaeb619e2a3331d2acc88464299c0c73e" exitCode=0 Jul 02 13:44:45.430015 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:45.429570 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b79bbc56f-sm94p" Jul 02 13:44:45.430015 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:45.429586 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b79bbc56f-sm94p" event={"ID":"fb83e970-4eb8-4518-bec4-e7127c3f21df","Type":"ContainerDied","Data":"4d7616c3099c7e665e48f78e8e1e56deaeb619e2a3331d2acc88464299c0c73e"} Jul 02 13:44:45.430015 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:45.429642 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b79bbc56f-sm94p" event={"ID":"fb83e970-4eb8-4518-bec4-e7127c3f21df","Type":"ContainerDied","Data":"04f3a68ee19f08b3de8887d073ce5a9f0ba811854e7cf84544b0a10843a485c2"} Jul 02 13:44:45.430015 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:45.429663 2561 scope.go:117] "RemoveContainer" containerID="4d7616c3099c7e665e48f78e8e1e56deaeb619e2a3331d2acc88464299c0c73e" Jul 02 13:44:45.441040 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:45.441019 2561 scope.go:117] "RemoveContainer" containerID="4d7616c3099c7e665e48f78e8e1e56deaeb619e2a3331d2acc88464299c0c73e" Jul 02 13:44:45.441478 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:44:45.441454 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d7616c3099c7e665e48f78e8e1e56deaeb619e2a3331d2acc88464299c0c73e\": container with ID starting with 4d7616c3099c7e665e48f78e8e1e56deaeb619e2a3331d2acc88464299c0c73e not found: ID does not exist" containerID="4d7616c3099c7e665e48f78e8e1e56deaeb619e2a3331d2acc88464299c0c73e" Jul 02 13:44:45.441565 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:45.441487 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d7616c3099c7e665e48f78e8e1e56deaeb619e2a3331d2acc88464299c0c73e"} err="failed to get container status \"4d7616c3099c7e665e48f78e8e1e56deaeb619e2a3331d2acc88464299c0c73e\": rpc error: code = NotFound desc = could not find container \"4d7616c3099c7e665e48f78e8e1e56deaeb619e2a3331d2acc88464299c0c73e\": container with ID starting with 4d7616c3099c7e665e48f78e8e1e56deaeb619e2a3331d2acc88464299c0c73e not found: ID does not exist" Jul 02 13:44:45.450936 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:45.450915 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b79bbc56f-sm94p"] Jul 02 13:44:45.454540 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:45.454519 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b79bbc56f-sm94p"] Jul 02 13:44:46.362887 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:44:46.362855 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb83e970-4eb8-4518-bec4-e7127c3f21df" path="/var/lib/kubelet/pods/fb83e970-4eb8-4518-bec4-e7127c3f21df/volumes" Jul 02 13:45:05.621911 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:05.621863 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:05.637991 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:05.637953 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:06.534562 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:06.534521 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 13:45:35.186692 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.186635 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56bfd54c55-jbczs"] Jul 02 13:45:35.187256 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.187232 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb83e970-4eb8-4518-bec4-e7127c3f21df" containerName="console" Jul 02 13:45:35.187256 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.187256 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb83e970-4eb8-4518-bec4-e7127c3f21df" containerName="console" Jul 02 13:45:35.187441 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.187332 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb83e970-4eb8-4518-bec4-e7127c3f21df" containerName="console" Jul 02 13:45:35.190856 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.190838 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.211595 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.211569 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56bfd54c55-jbczs"] Jul 02 13:45:35.229057 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.229026 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-service-ca\") pod \"console-56bfd54c55-jbczs\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.229256 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.229117 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/38e48c60-7ea2-4d2d-9b68-0474e66ce230-console-serving-cert\") pod \"console-56bfd54c55-jbczs\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.229256 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.229159 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-trusted-ca-bundle\") pod \"console-56bfd54c55-jbczs\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.229256 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.229186 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-oauth-serving-cert\") pod \"console-56bfd54c55-jbczs\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.229256 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.229232 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-console-config\") pod \"console-56bfd54c55-jbczs\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.229438 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.229258 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltppq\" (UniqueName: \"kubernetes.io/projected/38e48c60-7ea2-4d2d-9b68-0474e66ce230-kube-api-access-ltppq\") pod \"console-56bfd54c55-jbczs\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.229438 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.229307 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/38e48c60-7ea2-4d2d-9b68-0474e66ce230-console-oauth-config\") pod \"console-56bfd54c55-jbczs\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.330182 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.330142 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/38e48c60-7ea2-4d2d-9b68-0474e66ce230-console-serving-cert\") pod \"console-56bfd54c55-jbczs\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.330182 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.330187 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-trusted-ca-bundle\") pod \"console-56bfd54c55-jbczs\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.330454 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.330205 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-oauth-serving-cert\") pod \"console-56bfd54c55-jbczs\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.330454 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.330234 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-console-config\") pod \"console-56bfd54c55-jbczs\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.330454 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.330258 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltppq\" (UniqueName: \"kubernetes.io/projected/38e48c60-7ea2-4d2d-9b68-0474e66ce230-kube-api-access-ltppq\") pod \"console-56bfd54c55-jbczs\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.330454 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.330298 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/38e48c60-7ea2-4d2d-9b68-0474e66ce230-console-oauth-config\") pod \"console-56bfd54c55-jbczs\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.330454 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.330330 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-service-ca\") pod \"console-56bfd54c55-jbczs\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.331031 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.331004 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-oauth-serving-cert\") pod \"console-56bfd54c55-jbczs\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.331123 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.331093 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-service-ca\") pod \"console-56bfd54c55-jbczs\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.337218 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.331093 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-console-config\") pod \"console-56bfd54c55-jbczs\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.337218 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.331504 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-trusted-ca-bundle\") pod \"console-56bfd54c55-jbczs\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.338088 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.338066 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/38e48c60-7ea2-4d2d-9b68-0474e66ce230-console-serving-cert\") pod \"console-56bfd54c55-jbczs\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.338171 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.338145 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/38e48c60-7ea2-4d2d-9b68-0474e66ce230-console-oauth-config\") pod \"console-56bfd54c55-jbczs\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.339316 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.339292 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltppq\" (UniqueName: \"kubernetes.io/projected/38e48c60-7ea2-4d2d-9b68-0474e66ce230-kube-api-access-ltppq\") pod \"console-56bfd54c55-jbczs\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.504374 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.504336 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:35.641710 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:35.641647 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56bfd54c55-jbczs"] Jul 02 13:45:35.644543 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:45:35.644510 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38e48c60_7ea2_4d2d_9b68_0474e66ce230.slice/crio-0598231c9fe0388280a9844a90f69bb5de447ca3d4d54ded9501e749f2e6b889 WatchSource:0}: Error finding container 0598231c9fe0388280a9844a90f69bb5de447ca3d4d54ded9501e749f2e6b889: Status 404 returned error can't find the container with id 0598231c9fe0388280a9844a90f69bb5de447ca3d4d54ded9501e749f2e6b889 Jul 02 13:45:36.641697 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:36.641665 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56bfd54c55-jbczs" event={"ID":"38e48c60-7ea2-4d2d-9b68-0474e66ce230","Type":"ContainerStarted","Data":"b4ea533d54ac08af4db95b0df7b8d28b31d9b8ac0537bd7b588fad6658b2887a"} Jul 02 13:45:36.642326 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:36.641711 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56bfd54c55-jbczs" event={"ID":"38e48c60-7ea2-4d2d-9b68-0474e66ce230","Type":"ContainerStarted","Data":"0598231c9fe0388280a9844a90f69bb5de447ca3d4d54ded9501e749f2e6b889"} Jul 02 13:45:36.660388 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:36.660322 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56bfd54c55-jbczs" podStartSLOduration=1.660297798 podStartE2EDuration="1.660297798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:45:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:45:36.658325833 +0000 UTC m=+242.784462339" watchObservedRunningTime="2026-07-02 13:45:36.660297798 +0000 UTC m=+242.786434305" Jul 02 13:45:45.504778 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:45.504745 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:45.505188 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:45.504800 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:45.509365 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:45.509347 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:45.681195 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:45.681167 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:45:45.729773 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:45:45.729746 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8994dd95c-drhh8"] Jul 02 13:46:10.750110 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:10.750073 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-8994dd95c-drhh8" podUID="e1be8b6d-4dc5-4190-a2ce-142743bb3b71" containerName="console" containerID="cri-o://0cc500981f7739631440011244a7811af4837060a5046755df92e44caa487c32" gracePeriod=15 Jul 02 13:46:10.996403 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:10.996371 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:46:11.059547 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.059433 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-console-config\") pod \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " Jul 02 13:46:11.059547 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.059500 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-trusted-ca-bundle\") pod \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " Jul 02 13:46:11.059547 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.059519 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9d4x\" (UniqueName: \"kubernetes.io/projected/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-kube-api-access-n9d4x\") pod \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " Jul 02 13:46:11.059547 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.059554 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-console-oauth-config\") pod \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " Jul 02 13:46:11.059891 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.059577 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-oauth-serving-cert\") pod \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " Jul 02 13:46:11.059891 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.059591 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-console-serving-cert\") pod \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " Jul 02 13:46:11.059891 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.059645 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-service-ca\") pod \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\" (UID: \"e1be8b6d-4dc5-4190-a2ce-142743bb3b71\") " Jul 02 13:46:11.060076 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.059911 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-console-config" (OuterVolumeSpecName: "console-config") pod "e1be8b6d-4dc5-4190-a2ce-142743bb3b71" (UID: "e1be8b6d-4dc5-4190-a2ce-142743bb3b71"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:46:11.060076 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.060021 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e1be8b6d-4dc5-4190-a2ce-142743bb3b71" (UID: "e1be8b6d-4dc5-4190-a2ce-142743bb3b71"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:46:11.060076 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.060052 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e1be8b6d-4dc5-4190-a2ce-142743bb3b71" (UID: "e1be8b6d-4dc5-4190-a2ce-142743bb3b71"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:46:11.060216 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.060183 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-service-ca" (OuterVolumeSpecName: "service-ca") pod "e1be8b6d-4dc5-4190-a2ce-142743bb3b71" (UID: "e1be8b6d-4dc5-4190-a2ce-142743bb3b71"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:46:11.061792 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.061755 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-kube-api-access-n9d4x" (OuterVolumeSpecName: "kube-api-access-n9d4x") pod "e1be8b6d-4dc5-4190-a2ce-142743bb3b71" (UID: "e1be8b6d-4dc5-4190-a2ce-142743bb3b71"). InnerVolumeSpecName "kube-api-access-n9d4x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:46:11.061887 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.061808 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e1be8b6d-4dc5-4190-a2ce-142743bb3b71" (UID: "e1be8b6d-4dc5-4190-a2ce-142743bb3b71"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:46:11.061887 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.061831 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e1be8b6d-4dc5-4190-a2ce-142743bb3b71" (UID: "e1be8b6d-4dc5-4190-a2ce-142743bb3b71"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:46:11.160436 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.160399 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-trusted-ca-bundle\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:46:11.160436 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.160427 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9d4x\" (UniqueName: \"kubernetes.io/projected/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-kube-api-access-n9d4x\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:46:11.160436 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.160438 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-console-oauth-config\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:46:11.160436 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.160447 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-oauth-serving-cert\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:46:11.160689 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.160456 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-console-serving-cert\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:46:11.160689 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.160465 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-service-ca\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:46:11.160689 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.160473 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e1be8b6d-4dc5-4190-a2ce-142743bb3b71-console-config\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:46:11.787547 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.787512 2561 generic.go:356] "Generic (PLEG): container finished" podID="e1be8b6d-4dc5-4190-a2ce-142743bb3b71" containerID="0cc500981f7739631440011244a7811af4837060a5046755df92e44caa487c32" exitCode=0 Jul 02 13:46:11.788014 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.787557 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8994dd95c-drhh8" Jul 02 13:46:11.788014 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.787595 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8994dd95c-drhh8" event={"ID":"e1be8b6d-4dc5-4190-a2ce-142743bb3b71","Type":"ContainerDied","Data":"0cc500981f7739631440011244a7811af4837060a5046755df92e44caa487c32"} Jul 02 13:46:11.788014 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.787641 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8994dd95c-drhh8" event={"ID":"e1be8b6d-4dc5-4190-a2ce-142743bb3b71","Type":"ContainerDied","Data":"2c315d783c549809e3d2cc33472b7a076efae962596c92d2a4c0d6716ab98ca5"} Jul 02 13:46:11.788014 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.787665 2561 scope.go:117] "RemoveContainer" containerID="0cc500981f7739631440011244a7811af4837060a5046755df92e44caa487c32" Jul 02 13:46:11.796825 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.796808 2561 scope.go:117] "RemoveContainer" containerID="0cc500981f7739631440011244a7811af4837060a5046755df92e44caa487c32" Jul 02 13:46:11.797147 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:46:11.797128 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cc500981f7739631440011244a7811af4837060a5046755df92e44caa487c32\": container with ID starting with 0cc500981f7739631440011244a7811af4837060a5046755df92e44caa487c32 not found: ID does not exist" containerID="0cc500981f7739631440011244a7811af4837060a5046755df92e44caa487c32" Jul 02 13:46:11.797227 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.797154 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cc500981f7739631440011244a7811af4837060a5046755df92e44caa487c32"} err="failed to get container status \"0cc500981f7739631440011244a7811af4837060a5046755df92e44caa487c32\": rpc error: code = NotFound desc = could not find container \"0cc500981f7739631440011244a7811af4837060a5046755df92e44caa487c32\": container with ID starting with 0cc500981f7739631440011244a7811af4837060a5046755df92e44caa487c32 not found: ID does not exist" Jul 02 13:46:11.809160 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.809124 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8994dd95c-drhh8"] Jul 02 13:46:11.813228 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:11.813198 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8994dd95c-drhh8"] Jul 02 13:46:12.362950 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:12.362905 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1be8b6d-4dc5-4190-a2ce-142743bb3b71" path="/var/lib/kubelet/pods/e1be8b6d-4dc5-4190-a2ce-142743bb3b71/volumes" Jul 02 13:46:34.321406 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:34.321378 2561 kubelet.go:1648] "Image garbage collection succeeded" Jul 02 13:46:58.083402 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:58.083368 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-xj5vj"] Jul 02 13:46:58.085722 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:58.083701 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1be8b6d-4dc5-4190-a2ce-142743bb3b71" containerName="console" Jul 02 13:46:58.085722 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:58.083711 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1be8b6d-4dc5-4190-a2ce-142743bb3b71" containerName="console" Jul 02 13:46:58.085722 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:58.083777 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1be8b6d-4dc5-4190-a2ce-142743bb3b71" containerName="console" Jul 02 13:46:58.086516 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:58.086501 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-xj5vj" Jul 02 13:46:58.088780 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:58.088751 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-q6mqc\"" Jul 02 13:46:58.088903 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:58.088785 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 02 13:46:58.088903 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:58.088822 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 02 13:46:58.094146 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:58.094125 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-xj5vj"] Jul 02 13:46:58.182393 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:58.182351 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wgck\" (UniqueName: \"kubernetes.io/projected/5283e2e4-7014-440a-a301-e5f0f19e8d5a-kube-api-access-2wgck\") pod \"cert-manager-webhook-7dc76cbb94-xj5vj\" (UID: \"5283e2e4-7014-440a-a301-e5f0f19e8d5a\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-xj5vj" Jul 02 13:46:58.182573 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:58.182402 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5283e2e4-7014-440a-a301-e5f0f19e8d5a-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-xj5vj\" (UID: \"5283e2e4-7014-440a-a301-e5f0f19e8d5a\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-xj5vj" Jul 02 13:46:58.283027 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:58.282994 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wgck\" (UniqueName: \"kubernetes.io/projected/5283e2e4-7014-440a-a301-e5f0f19e8d5a-kube-api-access-2wgck\") pod \"cert-manager-webhook-7dc76cbb94-xj5vj\" (UID: \"5283e2e4-7014-440a-a301-e5f0f19e8d5a\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-xj5vj" Jul 02 13:46:58.283229 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:58.283043 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5283e2e4-7014-440a-a301-e5f0f19e8d5a-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-xj5vj\" (UID: \"5283e2e4-7014-440a-a301-e5f0f19e8d5a\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-xj5vj" Jul 02 13:46:58.293009 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:58.292948 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5283e2e4-7014-440a-a301-e5f0f19e8d5a-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-xj5vj\" (UID: \"5283e2e4-7014-440a-a301-e5f0f19e8d5a\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-xj5vj" Jul 02 13:46:58.293139 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:58.293037 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wgck\" (UniqueName: \"kubernetes.io/projected/5283e2e4-7014-440a-a301-e5f0f19e8d5a-kube-api-access-2wgck\") pod \"cert-manager-webhook-7dc76cbb94-xj5vj\" (UID: \"5283e2e4-7014-440a-a301-e5f0f19e8d5a\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-xj5vj" Jul 02 13:46:58.411270 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:58.411175 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-xj5vj" Jul 02 13:46:58.539684 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:58.539649 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-xj5vj"] Jul 02 13:46:58.542754 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:46:58.542720 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5283e2e4_7014_440a_a301_e5f0f19e8d5a.slice/crio-bd0574e52ecc433e433648c16dc4e1d5969ab1b841b753f4667b635b87b0624d WatchSource:0}: Error finding container bd0574e52ecc433e433648c16dc4e1d5969ab1b841b753f4667b635b87b0624d: Status 404 returned error can't find the container with id bd0574e52ecc433e433648c16dc4e1d5969ab1b841b753f4667b635b87b0624d Jul 02 13:46:58.544734 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:58.544717 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 13:46:58.984007 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:46:58.983956 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-xj5vj" event={"ID":"5283e2e4-7014-440a-a301-e5f0f19e8d5a","Type":"ContainerStarted","Data":"bd0574e52ecc433e433648c16dc4e1d5969ab1b841b753f4667b635b87b0624d"} Jul 02 13:47:02.000704 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:02.000665 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-xj5vj" event={"ID":"5283e2e4-7014-440a-a301-e5f0f19e8d5a","Type":"ContainerStarted","Data":"0c64d839b869c670e93a72bd774bfd1865407e115fb5a688192b692c48aa4c1e"} Jul 02 13:47:02.001145 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:02.000738 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-xj5vj" Jul 02 13:47:02.023266 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:02.023171 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-7dc76cbb94-xj5vj" podStartSLOduration=0.801624228 podStartE2EDuration="4.02315815s" totalImagesPullingTime="3.221533922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:46:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:47:02.022031152 +0000 UTC m=+328.148167659" watchObservedRunningTime="2026-07-02 13:47:02.02315815 +0000 UTC m=+328.149294655" Jul 02 13:47:08.007302 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:08.007262 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-xj5vj" Jul 02 13:47:30.223638 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:30.223547 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s8f4q"] Jul 02 13:47:30.227125 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:30.227104 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s8f4q" Jul 02 13:47:30.229621 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:30.229591 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 02 13:47:30.230421 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:30.230405 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-xtkjf\"" Jul 02 13:47:30.230421 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:30.230415 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 02 13:47:30.235159 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:30.235132 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s8f4q"] Jul 02 13:47:30.363354 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:30.363328 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mx9m\" (UniqueName: \"kubernetes.io/projected/24d69536-4028-4050-a6a3-5bdf03b57b7f-kube-api-access-7mx9m\") pod \"openshift-lws-operator-fd8ccff4c-s8f4q\" (UID: \"24d69536-4028-4050-a6a3-5bdf03b57b7f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s8f4q" Jul 02 13:47:30.363524 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:30.363363 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24d69536-4028-4050-a6a3-5bdf03b57b7f-tmp\") pod \"openshift-lws-operator-fd8ccff4c-s8f4q\" (UID: \"24d69536-4028-4050-a6a3-5bdf03b57b7f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s8f4q" Jul 02 13:47:30.464753 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:30.464714 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mx9m\" (UniqueName: \"kubernetes.io/projected/24d69536-4028-4050-a6a3-5bdf03b57b7f-kube-api-access-7mx9m\") pod \"openshift-lws-operator-fd8ccff4c-s8f4q\" (UID: \"24d69536-4028-4050-a6a3-5bdf03b57b7f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s8f4q" Jul 02 13:47:30.464753 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:30.464759 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24d69536-4028-4050-a6a3-5bdf03b57b7f-tmp\") pod \"openshift-lws-operator-fd8ccff4c-s8f4q\" (UID: \"24d69536-4028-4050-a6a3-5bdf03b57b7f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s8f4q" Jul 02 13:47:30.465165 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:30.465148 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24d69536-4028-4050-a6a3-5bdf03b57b7f-tmp\") pod \"openshift-lws-operator-fd8ccff4c-s8f4q\" (UID: \"24d69536-4028-4050-a6a3-5bdf03b57b7f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s8f4q" Jul 02 13:47:30.475363 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:30.475299 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mx9m\" (UniqueName: \"kubernetes.io/projected/24d69536-4028-4050-a6a3-5bdf03b57b7f-kube-api-access-7mx9m\") pod \"openshift-lws-operator-fd8ccff4c-s8f4q\" (UID: \"24d69536-4028-4050-a6a3-5bdf03b57b7f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s8f4q" Jul 02 13:47:30.536902 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:30.536865 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s8f4q" Jul 02 13:47:30.665193 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:30.665167 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s8f4q"] Jul 02 13:47:30.667928 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:47:30.667900 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24d69536_4028_4050_a6a3_5bdf03b57b7f.slice/crio-4fb5ae956b8062e204557fd605945b6eb4aed0a2d283ca047dfd5aeb322f40aa WatchSource:0}: Error finding container 4fb5ae956b8062e204557fd605945b6eb4aed0a2d283ca047dfd5aeb322f40aa: Status 404 returned error can't find the container with id 4fb5ae956b8062e204557fd605945b6eb4aed0a2d283ca047dfd5aeb322f40aa Jul 02 13:47:31.130244 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:31.130207 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s8f4q" event={"ID":"24d69536-4028-4050-a6a3-5bdf03b57b7f","Type":"ContainerStarted","Data":"4fb5ae956b8062e204557fd605945b6eb4aed0a2d283ca047dfd5aeb322f40aa"} Jul 02 13:47:33.140643 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:33.140608 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s8f4q" event={"ID":"24d69536-4028-4050-a6a3-5bdf03b57b7f","Type":"ContainerStarted","Data":"5af8a87491853a80b5b8a41606ddb83f270c7498faa6bf858fa9735602c1d792"} Jul 02 13:47:33.165376 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:33.165260 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-s8f4q" podStartSLOduration=0.940893012 podStartE2EDuration="3.165242066s" totalImagesPullingTime="2.224349054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:47:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:47:33.163165113 +0000 UTC m=+359.289301630" watchObservedRunningTime="2026-07-02 13:47:33.165242066 +0000 UTC m=+359.291378572" Jul 02 13:47:40.140381 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:40.140347 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn"] Jul 02 13:47:40.144959 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:40.144932 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn" Jul 02 13:47:40.147292 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:40.147260 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 02 13:47:40.147955 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:40.147936 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 02 13:47:40.148069 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:40.147938 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-5lwdn\"" Jul 02 13:47:40.148069 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:40.148016 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 02 13:47:40.155219 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:40.155193 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn"] Jul 02 13:47:40.252521 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:40.252476 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h96qq\" (UniqueName: \"kubernetes.io/projected/2f680c7f-2f26-45f5-ae98-89eb89c8df42-kube-api-access-h96qq\") pod \"lws-controller-manager-7497c854c6-tlnxn\" (UID: \"2f680c7f-2f26-45f5-ae98-89eb89c8df42\") " pod="openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn" Jul 02 13:47:40.252730 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:40.252602 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2f680c7f-2f26-45f5-ae98-89eb89c8df42-manager-config\") pod \"lws-controller-manager-7497c854c6-tlnxn\" (UID: \"2f680c7f-2f26-45f5-ae98-89eb89c8df42\") " pod="openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn" Jul 02 13:47:40.252730 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:40.252644 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2f680c7f-2f26-45f5-ae98-89eb89c8df42-metrics-cert\") pod \"lws-controller-manager-7497c854c6-tlnxn\" (UID: \"2f680c7f-2f26-45f5-ae98-89eb89c8df42\") " pod="openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn" Jul 02 13:47:40.252730 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:40.252674 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f680c7f-2f26-45f5-ae98-89eb89c8df42-cert\") pod \"lws-controller-manager-7497c854c6-tlnxn\" (UID: \"2f680c7f-2f26-45f5-ae98-89eb89c8df42\") " pod="openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn" Jul 02 13:47:40.354003 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:40.353938 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2f680c7f-2f26-45f5-ae98-89eb89c8df42-manager-config\") pod \"lws-controller-manager-7497c854c6-tlnxn\" (UID: \"2f680c7f-2f26-45f5-ae98-89eb89c8df42\") " pod="openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn" Jul 02 13:47:40.354197 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:40.354012 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2f680c7f-2f26-45f5-ae98-89eb89c8df42-metrics-cert\") pod \"lws-controller-manager-7497c854c6-tlnxn\" (UID: \"2f680c7f-2f26-45f5-ae98-89eb89c8df42\") " pod="openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn" Jul 02 13:47:40.354197 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:40.354044 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f680c7f-2f26-45f5-ae98-89eb89c8df42-cert\") pod \"lws-controller-manager-7497c854c6-tlnxn\" (UID: \"2f680c7f-2f26-45f5-ae98-89eb89c8df42\") " pod="openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn" Jul 02 13:47:40.354197 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:40.354101 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h96qq\" (UniqueName: \"kubernetes.io/projected/2f680c7f-2f26-45f5-ae98-89eb89c8df42-kube-api-access-h96qq\") pod \"lws-controller-manager-7497c854c6-tlnxn\" (UID: \"2f680c7f-2f26-45f5-ae98-89eb89c8df42\") " pod="openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn" Jul 02 13:47:40.354590 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:40.354567 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2f680c7f-2f26-45f5-ae98-89eb89c8df42-manager-config\") pod \"lws-controller-manager-7497c854c6-tlnxn\" (UID: \"2f680c7f-2f26-45f5-ae98-89eb89c8df42\") " pod="openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn" Jul 02 13:47:40.356529 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:40.356511 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2f680c7f-2f26-45f5-ae98-89eb89c8df42-metrics-cert\") pod \"lws-controller-manager-7497c854c6-tlnxn\" (UID: \"2f680c7f-2f26-45f5-ae98-89eb89c8df42\") " pod="openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn" Jul 02 13:47:40.356614 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:40.356599 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2f680c7f-2f26-45f5-ae98-89eb89c8df42-cert\") pod \"lws-controller-manager-7497c854c6-tlnxn\" (UID: \"2f680c7f-2f26-45f5-ae98-89eb89c8df42\") " pod="openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn" Jul 02 13:47:40.362493 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:40.362471 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h96qq\" (UniqueName: \"kubernetes.io/projected/2f680c7f-2f26-45f5-ae98-89eb89c8df42-kube-api-access-h96qq\") pod \"lws-controller-manager-7497c854c6-tlnxn\" (UID: \"2f680c7f-2f26-45f5-ae98-89eb89c8df42\") " pod="openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn" Jul 02 13:47:40.455499 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:40.455396 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn" Jul 02 13:47:40.597521 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:40.597493 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn"] Jul 02 13:47:40.600273 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:47:40.600247 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f680c7f_2f26_45f5_ae98_89eb89c8df42.slice/crio-a769ec7502e52f8fc1cfda47ba1ea3c8639cd69c495580b3966b856bee50d7ee WatchSource:0}: Error finding container a769ec7502e52f8fc1cfda47ba1ea3c8639cd69c495580b3966b856bee50d7ee: Status 404 returned error can't find the container with id a769ec7502e52f8fc1cfda47ba1ea3c8639cd69c495580b3966b856bee50d7ee Jul 02 13:47:41.178108 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:41.178030 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn" event={"ID":"2f680c7f-2f26-45f5-ae98-89eb89c8df42","Type":"ContainerStarted","Data":"a769ec7502e52f8fc1cfda47ba1ea3c8639cd69c495580b3966b856bee50d7ee"} Jul 02 13:47:42.183186 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:42.183150 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn" event={"ID":"2f680c7f-2f26-45f5-ae98-89eb89c8df42","Type":"ContainerStarted","Data":"9440ca5bc7e6decdd7408bc2b8bca6c001ba04f77f14c3838eb92b69caff8942"} Jul 02 13:47:42.183552 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:42.183241 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn" Jul 02 13:47:42.199762 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:42.199714 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn" podStartSLOduration=0.82034469 podStartE2EDuration="2.199698181s" totalImagesPullingTime="1.379353491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:47:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:47:42.197739241 +0000 UTC m=+368.323875747" watchObservedRunningTime="2026-07-02 13:47:42.199698181 +0000 UTC m=+368.325834686" Jul 02 13:47:52.401826 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.401790 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz"] Jul 02 13:47:52.404567 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.404546 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" Jul 02 13:47:52.407011 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.406965 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6vkzj\"" Jul 02 13:47:52.407133 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.407113 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 02 13:47:52.407244 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.407210 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 02 13:47:52.407453 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.407439 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 02 13:47:52.407586 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.407570 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 02 13:47:52.420770 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.420749 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz"] Jul 02 13:47:52.464888 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.464852 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f18bedb-22f1-43db-be9e-a7012ff27142-tmp\") pod \"opendatahub-operator-controller-manager-5964c6974c-sfjzz\" (UID: \"9f18bedb-22f1-43db-be9e-a7012ff27142\") " pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" Jul 02 13:47:52.465072 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.464904 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9f18bedb-22f1-43db-be9e-a7012ff27142-manifests\") pod \"opendatahub-operator-controller-manager-5964c6974c-sfjzz\" (UID: \"9f18bedb-22f1-43db-be9e-a7012ff27142\") " pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" Jul 02 13:47:52.465072 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.464961 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9f18bedb-22f1-43db-be9e-a7012ff27142-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5964c6974c-sfjzz\" (UID: \"9f18bedb-22f1-43db-be9e-a7012ff27142\") " pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" Jul 02 13:47:52.465072 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.465056 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f18bedb-22f1-43db-be9e-a7012ff27142-webhook-cert\") pod \"opendatahub-operator-controller-manager-5964c6974c-sfjzz\" (UID: \"9f18bedb-22f1-43db-be9e-a7012ff27142\") " pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" Jul 02 13:47:52.465181 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.465087 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chdth\" (UniqueName: \"kubernetes.io/projected/9f18bedb-22f1-43db-be9e-a7012ff27142-kube-api-access-chdth\") pod \"opendatahub-operator-controller-manager-5964c6974c-sfjzz\" (UID: \"9f18bedb-22f1-43db-be9e-a7012ff27142\") " pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" Jul 02 13:47:52.566170 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.566126 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9f18bedb-22f1-43db-be9e-a7012ff27142-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5964c6974c-sfjzz\" (UID: \"9f18bedb-22f1-43db-be9e-a7012ff27142\") " pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" Jul 02 13:47:52.566170 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.566183 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f18bedb-22f1-43db-be9e-a7012ff27142-webhook-cert\") pod \"opendatahub-operator-controller-manager-5964c6974c-sfjzz\" (UID: \"9f18bedb-22f1-43db-be9e-a7012ff27142\") " pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" Jul 02 13:47:52.566420 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.566206 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chdth\" (UniqueName: \"kubernetes.io/projected/9f18bedb-22f1-43db-be9e-a7012ff27142-kube-api-access-chdth\") pod \"opendatahub-operator-controller-manager-5964c6974c-sfjzz\" (UID: \"9f18bedb-22f1-43db-be9e-a7012ff27142\") " pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" Jul 02 13:47:52.566420 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.566259 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f18bedb-22f1-43db-be9e-a7012ff27142-tmp\") pod \"opendatahub-operator-controller-manager-5964c6974c-sfjzz\" (UID: \"9f18bedb-22f1-43db-be9e-a7012ff27142\") " pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" Jul 02 13:47:52.566509 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.566454 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9f18bedb-22f1-43db-be9e-a7012ff27142-manifests\") pod \"opendatahub-operator-controller-manager-5964c6974c-sfjzz\" (UID: \"9f18bedb-22f1-43db-be9e-a7012ff27142\") " pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" Jul 02 13:47:52.566768 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.566748 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9f18bedb-22f1-43db-be9e-a7012ff27142-manifests\") pod \"opendatahub-operator-controller-manager-5964c6974c-sfjzz\" (UID: \"9f18bedb-22f1-43db-be9e-a7012ff27142\") " pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" Jul 02 13:47:52.568526 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.568504 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9f18bedb-22f1-43db-be9e-a7012ff27142-tmp\") pod \"opendatahub-operator-controller-manager-5964c6974c-sfjzz\" (UID: \"9f18bedb-22f1-43db-be9e-a7012ff27142\") " pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" Jul 02 13:47:52.568653 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.568637 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9f18bedb-22f1-43db-be9e-a7012ff27142-webhook-cert\") pod \"opendatahub-operator-controller-manager-5964c6974c-sfjzz\" (UID: \"9f18bedb-22f1-43db-be9e-a7012ff27142\") " pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" Jul 02 13:47:52.568718 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.568700 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9f18bedb-22f1-43db-be9e-a7012ff27142-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5964c6974c-sfjzz\" (UID: \"9f18bedb-22f1-43db-be9e-a7012ff27142\") " pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" Jul 02 13:47:52.574098 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.574074 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chdth\" (UniqueName: \"kubernetes.io/projected/9f18bedb-22f1-43db-be9e-a7012ff27142-kube-api-access-chdth\") pod \"opendatahub-operator-controller-manager-5964c6974c-sfjzz\" (UID: \"9f18bedb-22f1-43db-be9e-a7012ff27142\") " pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" Jul 02 13:47:52.715397 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.715365 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" Jul 02 13:47:52.854408 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:52.854377 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz"] Jul 02 13:47:52.857920 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:47:52.857894 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f18bedb_22f1_43db_be9e_a7012ff27142.slice/crio-3e63996ae344bb51f52f89e374defc27d2f130d27f1ae735a1b919bd88a16547 WatchSource:0}: Error finding container 3e63996ae344bb51f52f89e374defc27d2f130d27f1ae735a1b919bd88a16547: Status 404 returned error can't find the container with id 3e63996ae344bb51f52f89e374defc27d2f130d27f1ae735a1b919bd88a16547 Jul 02 13:47:53.189838 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:53.189745 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7497c854c6-tlnxn" Jul 02 13:47:53.233165 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:53.233123 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" event={"ID":"9f18bedb-22f1-43db-be9e-a7012ff27142","Type":"ContainerStarted","Data":"3e63996ae344bb51f52f89e374defc27d2f130d27f1ae735a1b919bd88a16547"} Jul 02 13:47:56.248938 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:56.248899 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" event={"ID":"9f18bedb-22f1-43db-be9e-a7012ff27142","Type":"ContainerStarted","Data":"3d8e59db33b3a3d5e1b79a6cea2a46b70b091e3642fb813c395c05a8598d31d9"} Jul 02 13:47:57.255020 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:57.254963 2561 generic.go:356] "Generic (PLEG): container finished" podID="9f18bedb-22f1-43db-be9e-a7012ff27142" containerID="3d8e59db33b3a3d5e1b79a6cea2a46b70b091e3642fb813c395c05a8598d31d9" exitCode=0 Jul 02 13:47:57.255469 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:57.255062 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" event={"ID":"9f18bedb-22f1-43db-be9e-a7012ff27142","Type":"ContainerDied","Data":"3d8e59db33b3a3d5e1b79a6cea2a46b70b091e3642fb813c395c05a8598d31d9"} Jul 02 13:47:58.261887 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:58.261854 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" event={"ID":"9f18bedb-22f1-43db-be9e-a7012ff27142","Type":"ContainerStarted","Data":"2782e0fa824682f5c790e2da68095e6e2fb494dda2b7734cb2ef1a9a896cee19"} Jul 02 13:47:58.262353 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:58.262116 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" Jul 02 13:47:58.288654 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:47:58.288587 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" podStartSLOduration=1.593209158 podStartE2EDuration="6.28857015s" totalImagesPullingTime="3.615476534s" totalInitContainerRuntime="1.079884458s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:47:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:47:58.287142581 +0000 UTC m=+384.413279084" watchObservedRunningTime="2026-07-02 13:47:58.28857015 +0000 UTC m=+384.414706654" Jul 02 13:48:09.269501 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:09.269472 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5964c6974c-sfjzz" Jul 02 13:48:12.482523 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:12.482487 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-685667f9f9-4rcj4"] Jul 02 13:48:12.489295 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:12.489271 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-685667f9f9-4rcj4" Jul 02 13:48:12.491447 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:12.491421 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 02 13:48:12.491716 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:12.491690 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xhzh6\"" Jul 02 13:48:12.491877 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:12.491861 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 02 13:48:12.491964 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:12.491907 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 02 13:48:12.491964 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:12.491923 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 02 13:48:12.506807 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:12.506785 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-685667f9f9-4rcj4"] Jul 02 13:48:12.647517 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:12.647481 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5698427-1618-4369-9d7c-82c74c7a27d1-tmp\") pod \"kube-auth-proxy-685667f9f9-4rcj4\" (UID: \"c5698427-1618-4369-9d7c-82c74c7a27d1\") " pod="openshift-ingress/kube-auth-proxy-685667f9f9-4rcj4" Jul 02 13:48:12.647696 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:12.647578 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c6kl\" (UniqueName: \"kubernetes.io/projected/c5698427-1618-4369-9d7c-82c74c7a27d1-kube-api-access-4c6kl\") pod \"kube-auth-proxy-685667f9f9-4rcj4\" (UID: \"c5698427-1618-4369-9d7c-82c74c7a27d1\") " pod="openshift-ingress/kube-auth-proxy-685667f9f9-4rcj4" Jul 02 13:48:12.647696 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:12.647615 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5698427-1618-4369-9d7c-82c74c7a27d1-tls-certs\") pod \"kube-auth-proxy-685667f9f9-4rcj4\" (UID: \"c5698427-1618-4369-9d7c-82c74c7a27d1\") " pod="openshift-ingress/kube-auth-proxy-685667f9f9-4rcj4" Jul 02 13:48:12.748736 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:12.748702 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5698427-1618-4369-9d7c-82c74c7a27d1-tmp\") pod \"kube-auth-proxy-685667f9f9-4rcj4\" (UID: \"c5698427-1618-4369-9d7c-82c74c7a27d1\") " pod="openshift-ingress/kube-auth-proxy-685667f9f9-4rcj4" Jul 02 13:48:12.748891 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:12.748769 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c6kl\" (UniqueName: \"kubernetes.io/projected/c5698427-1618-4369-9d7c-82c74c7a27d1-kube-api-access-4c6kl\") pod \"kube-auth-proxy-685667f9f9-4rcj4\" (UID: \"c5698427-1618-4369-9d7c-82c74c7a27d1\") " pod="openshift-ingress/kube-auth-proxy-685667f9f9-4rcj4" Jul 02 13:48:12.748891 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:12.748789 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5698427-1618-4369-9d7c-82c74c7a27d1-tls-certs\") pod \"kube-auth-proxy-685667f9f9-4rcj4\" (UID: \"c5698427-1618-4369-9d7c-82c74c7a27d1\") " pod="openshift-ingress/kube-auth-proxy-685667f9f9-4rcj4" Jul 02 13:48:12.751094 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:12.751065 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c5698427-1618-4369-9d7c-82c74c7a27d1-tmp\") pod \"kube-auth-proxy-685667f9f9-4rcj4\" (UID: \"c5698427-1618-4369-9d7c-82c74c7a27d1\") " pod="openshift-ingress/kube-auth-proxy-685667f9f9-4rcj4" Jul 02 13:48:12.751410 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:12.751388 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c5698427-1618-4369-9d7c-82c74c7a27d1-tls-certs\") pod \"kube-auth-proxy-685667f9f9-4rcj4\" (UID: \"c5698427-1618-4369-9d7c-82c74c7a27d1\") " pod="openshift-ingress/kube-auth-proxy-685667f9f9-4rcj4" Jul 02 13:48:12.760890 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:12.760863 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c6kl\" (UniqueName: \"kubernetes.io/projected/c5698427-1618-4369-9d7c-82c74c7a27d1-kube-api-access-4c6kl\") pod \"kube-auth-proxy-685667f9f9-4rcj4\" (UID: \"c5698427-1618-4369-9d7c-82c74c7a27d1\") " pod="openshift-ingress/kube-auth-proxy-685667f9f9-4rcj4" Jul 02 13:48:12.799713 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:12.799669 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-685667f9f9-4rcj4" Jul 02 13:48:12.935890 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:12.935866 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-685667f9f9-4rcj4"] Jul 02 13:48:12.938429 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:48:12.938401 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5698427_1618_4369_9d7c_82c74c7a27d1.slice/crio-4fb4b83749027cd915dae830035707a676dec57d8de80d9b46b5b92b7b8001bf WatchSource:0}: Error finding container 4fb4b83749027cd915dae830035707a676dec57d8de80d9b46b5b92b7b8001bf: Status 404 returned error can't find the container with id 4fb4b83749027cd915dae830035707a676dec57d8de80d9b46b5b92b7b8001bf Jul 02 13:48:13.331055 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:13.331021 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-685667f9f9-4rcj4" event={"ID":"c5698427-1618-4369-9d7c-82c74c7a27d1","Type":"ContainerStarted","Data":"4fb4b83749027cd915dae830035707a676dec57d8de80d9b46b5b92b7b8001bf"} Jul 02 13:48:17.351475 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:17.351443 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-685667f9f9-4rcj4" event={"ID":"c5698427-1618-4369-9d7c-82c74c7a27d1","Type":"ContainerStarted","Data":"f93f3f40216c93e8463dd962e84d93fe2b52df2198e63dda87dac1e8bc361d59"} Jul 02 13:48:17.368678 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:48:17.368619 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-685667f9f9-4rcj4" podStartSLOduration=1.939495554 podStartE2EDuration="5.368589827s" totalImagesPullingTime="3.429094273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:48:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:48:17.36732073 +0000 UTC m=+403.493457232" watchObservedRunningTime="2026-07-02 13:48:17.368589827 +0000 UTC m=+403.494726331" Jul 02 13:49:10.420242 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.420157 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-755c49bf7-js9r8"] Jul 02 13:49:10.423631 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.423610 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.442276 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.442249 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-755c49bf7-js9r8"] Jul 02 13:49:10.575827 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.575771 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8cc75d63-ff98-4268-ab92-b74dddfb4bd8-service-ca\") pod \"console-755c49bf7-js9r8\" (UID: \"8cc75d63-ff98-4268-ab92-b74dddfb4bd8\") " pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.576054 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.575849 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cc75d63-ff98-4268-ab92-b74dddfb4bd8-trusted-ca-bundle\") pod \"console-755c49bf7-js9r8\" (UID: \"8cc75d63-ff98-4268-ab92-b74dddfb4bd8\") " pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.576054 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.575876 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8cc75d63-ff98-4268-ab92-b74dddfb4bd8-console-oauth-config\") pod \"console-755c49bf7-js9r8\" (UID: \"8cc75d63-ff98-4268-ab92-b74dddfb4bd8\") " pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.576054 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.575896 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zmvc\" (UniqueName: \"kubernetes.io/projected/8cc75d63-ff98-4268-ab92-b74dddfb4bd8-kube-api-access-8zmvc\") pod \"console-755c49bf7-js9r8\" (UID: \"8cc75d63-ff98-4268-ab92-b74dddfb4bd8\") " pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.576054 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.575913 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8cc75d63-ff98-4268-ab92-b74dddfb4bd8-oauth-serving-cert\") pod \"console-755c49bf7-js9r8\" (UID: \"8cc75d63-ff98-4268-ab92-b74dddfb4bd8\") " pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.576213 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.576064 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8cc75d63-ff98-4268-ab92-b74dddfb4bd8-console-serving-cert\") pod \"console-755c49bf7-js9r8\" (UID: \"8cc75d63-ff98-4268-ab92-b74dddfb4bd8\") " pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.576213 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.576104 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8cc75d63-ff98-4268-ab92-b74dddfb4bd8-console-config\") pod \"console-755c49bf7-js9r8\" (UID: \"8cc75d63-ff98-4268-ab92-b74dddfb4bd8\") " pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.677063 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.676960 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8cc75d63-ff98-4268-ab92-b74dddfb4bd8-console-serving-cert\") pod \"console-755c49bf7-js9r8\" (UID: \"8cc75d63-ff98-4268-ab92-b74dddfb4bd8\") " pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.677063 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.677023 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8cc75d63-ff98-4268-ab92-b74dddfb4bd8-console-config\") pod \"console-755c49bf7-js9r8\" (UID: \"8cc75d63-ff98-4268-ab92-b74dddfb4bd8\") " pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.677280 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.677069 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8cc75d63-ff98-4268-ab92-b74dddfb4bd8-service-ca\") pod \"console-755c49bf7-js9r8\" (UID: \"8cc75d63-ff98-4268-ab92-b74dddfb4bd8\") " pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.677280 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.677092 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cc75d63-ff98-4268-ab92-b74dddfb4bd8-trusted-ca-bundle\") pod \"console-755c49bf7-js9r8\" (UID: \"8cc75d63-ff98-4268-ab92-b74dddfb4bd8\") " pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.677280 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.677113 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8cc75d63-ff98-4268-ab92-b74dddfb4bd8-console-oauth-config\") pod \"console-755c49bf7-js9r8\" (UID: \"8cc75d63-ff98-4268-ab92-b74dddfb4bd8\") " pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.677280 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.677131 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zmvc\" (UniqueName: \"kubernetes.io/projected/8cc75d63-ff98-4268-ab92-b74dddfb4bd8-kube-api-access-8zmvc\") pod \"console-755c49bf7-js9r8\" (UID: \"8cc75d63-ff98-4268-ab92-b74dddfb4bd8\") " pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.677280 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.677152 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8cc75d63-ff98-4268-ab92-b74dddfb4bd8-oauth-serving-cert\") pod \"console-755c49bf7-js9r8\" (UID: \"8cc75d63-ff98-4268-ab92-b74dddfb4bd8\") " pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.678323 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.678290 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8cc75d63-ff98-4268-ab92-b74dddfb4bd8-service-ca\") pod \"console-755c49bf7-js9r8\" (UID: \"8cc75d63-ff98-4268-ab92-b74dddfb4bd8\") " pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.678496 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.678477 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8cc75d63-ff98-4268-ab92-b74dddfb4bd8-oauth-serving-cert\") pod \"console-755c49bf7-js9r8\" (UID: \"8cc75d63-ff98-4268-ab92-b74dddfb4bd8\") " pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.678592 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.678542 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cc75d63-ff98-4268-ab92-b74dddfb4bd8-trusted-ca-bundle\") pod \"console-755c49bf7-js9r8\" (UID: \"8cc75d63-ff98-4268-ab92-b74dddfb4bd8\") " pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.680612 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.680584 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8cc75d63-ff98-4268-ab92-b74dddfb4bd8-console-oauth-config\") pod \"console-755c49bf7-js9r8\" (UID: \"8cc75d63-ff98-4268-ab92-b74dddfb4bd8\") " pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.680728 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.680576 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8cc75d63-ff98-4268-ab92-b74dddfb4bd8-console-serving-cert\") pod \"console-755c49bf7-js9r8\" (UID: \"8cc75d63-ff98-4268-ab92-b74dddfb4bd8\") " pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.686047 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.686017 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8cc75d63-ff98-4268-ab92-b74dddfb4bd8-console-config\") pod \"console-755c49bf7-js9r8\" (UID: \"8cc75d63-ff98-4268-ab92-b74dddfb4bd8\") " pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.687239 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.687209 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zmvc\" (UniqueName: \"kubernetes.io/projected/8cc75d63-ff98-4268-ab92-b74dddfb4bd8-kube-api-access-8zmvc\") pod \"console-755c49bf7-js9r8\" (UID: \"8cc75d63-ff98-4268-ab92-b74dddfb4bd8\") " pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.732707 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.732666 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:10.877051 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:10.877018 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-755c49bf7-js9r8"] Jul 02 13:49:10.880279 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:49:10.880252 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cc75d63_ff98_4268_ab92_b74dddfb4bd8.slice/crio-fd37a06f484cfa2dc43e997c285ceb640369431b45d3bd943f3c654721d7f70f WatchSource:0}: Error finding container fd37a06f484cfa2dc43e997c285ceb640369431b45d3bd943f3c654721d7f70f: Status 404 returned error can't find the container with id fd37a06f484cfa2dc43e997c285ceb640369431b45d3bd943f3c654721d7f70f Jul 02 13:49:11.609702 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:11.609666 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-755c49bf7-js9r8" event={"ID":"8cc75d63-ff98-4268-ab92-b74dddfb4bd8","Type":"ContainerStarted","Data":"8f4534c98740217b177974757b99f3dad58b31d822fca41f88ba52da228a3c19"} Jul 02 13:49:11.610105 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:11.609721 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-755c49bf7-js9r8" event={"ID":"8cc75d63-ff98-4268-ab92-b74dddfb4bd8","Type":"ContainerStarted","Data":"fd37a06f484cfa2dc43e997c285ceb640369431b45d3bd943f3c654721d7f70f"} Jul 02 13:49:20.733452 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:20.733413 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:20.733941 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:20.733741 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:20.738317 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:20.738293 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:20.758393 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:20.758343 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-755c49bf7-js9r8" podStartSLOduration=10.758323998 podStartE2EDuration="10.758323998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:49:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:49:11.632813404 +0000 UTC m=+457.758949911" watchObservedRunningTime="2026-07-02 13:49:20.758323998 +0000 UTC m=+466.884460507" Jul 02 13:49:21.664120 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:21.664091 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-755c49bf7-js9r8" Jul 02 13:49:21.750162 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:21.750128 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56bfd54c55-jbczs"] Jul 02 13:49:46.770802 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:46.770739 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-56bfd54c55-jbczs" podUID="38e48c60-7ea2-4d2d-9b68-0474e66ce230" containerName="console" containerID="cri-o://b4ea533d54ac08af4db95b0df7b8d28b31d9b8ac0537bd7b588fad6658b2887a" gracePeriod=15 Jul 02 13:49:47.018634 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.018609 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:49:47.122530 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.122441 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/38e48c60-7ea2-4d2d-9b68-0474e66ce230-console-serving-cert\") pod \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " Jul 02 13:49:47.122530 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.122481 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/38e48c60-7ea2-4d2d-9b68-0474e66ce230-console-oauth-config\") pod \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " Jul 02 13:49:47.122759 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.122539 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-oauth-serving-cert\") pod \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " Jul 02 13:49:47.122759 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.122561 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-console-config\") pod \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " Jul 02 13:49:47.122759 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.122631 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-trusted-ca-bundle\") pod \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " Jul 02 13:49:47.122759 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.122683 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-service-ca\") pod \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " Jul 02 13:49:47.123495 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.122937 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "38e48c60-7ea2-4d2d-9b68-0474e66ce230" (UID: "38e48c60-7ea2-4d2d-9b68-0474e66ce230"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:49:47.123495 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.123082 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ltppq\" (UniqueName: \"kubernetes.io/projected/38e48c60-7ea2-4d2d-9b68-0474e66ce230-kube-api-access-ltppq\") pod \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\" (UID: \"38e48c60-7ea2-4d2d-9b68-0474e66ce230\") " Jul 02 13:49:47.123495 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.123159 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-service-ca" (OuterVolumeSpecName: "service-ca") pod "38e48c60-7ea2-4d2d-9b68-0474e66ce230" (UID: "38e48c60-7ea2-4d2d-9b68-0474e66ce230"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:49:47.123495 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.123114 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-console-config" (OuterVolumeSpecName: "console-config") pod "38e48c60-7ea2-4d2d-9b68-0474e66ce230" (UID: "38e48c60-7ea2-4d2d-9b68-0474e66ce230"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:49:47.123495 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.123344 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "38e48c60-7ea2-4d2d-9b68-0474e66ce230" (UID: "38e48c60-7ea2-4d2d-9b68-0474e66ce230"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:49:47.123495 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.123480 2561 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-oauth-serving-cert\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:49:47.124047 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.123501 2561 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-console-config\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:49:47.124047 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.123518 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-trusted-ca-bundle\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:49:47.124047 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.123532 2561 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/38e48c60-7ea2-4d2d-9b68-0474e66ce230-service-ca\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:49:47.124830 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.124810 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38e48c60-7ea2-4d2d-9b68-0474e66ce230-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "38e48c60-7ea2-4d2d-9b68-0474e66ce230" (UID: "38e48c60-7ea2-4d2d-9b68-0474e66ce230"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:49:47.125497 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.125472 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38e48c60-7ea2-4d2d-9b68-0474e66ce230-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "38e48c60-7ea2-4d2d-9b68-0474e66ce230" (UID: "38e48c60-7ea2-4d2d-9b68-0474e66ce230"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:49:47.125561 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.125483 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38e48c60-7ea2-4d2d-9b68-0474e66ce230-kube-api-access-ltppq" (OuterVolumeSpecName: "kube-api-access-ltppq") pod "38e48c60-7ea2-4d2d-9b68-0474e66ce230" (UID: "38e48c60-7ea2-4d2d-9b68-0474e66ce230"). InnerVolumeSpecName "kube-api-access-ltppq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:49:47.224624 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.224579 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ltppq\" (UniqueName: \"kubernetes.io/projected/38e48c60-7ea2-4d2d-9b68-0474e66ce230-kube-api-access-ltppq\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:49:47.224624 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.224620 2561 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/38e48c60-7ea2-4d2d-9b68-0474e66ce230-console-serving-cert\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:49:47.224854 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.224635 2561 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/38e48c60-7ea2-4d2d-9b68-0474e66ce230-console-oauth-config\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:49:47.783888 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.783849 2561 generic.go:356] "Generic (PLEG): container finished" podID="38e48c60-7ea2-4d2d-9b68-0474e66ce230" containerID="b4ea533d54ac08af4db95b0df7b8d28b31d9b8ac0537bd7b588fad6658b2887a" exitCode=0 Jul 02 13:49:47.784306 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.783898 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56bfd54c55-jbczs" Jul 02 13:49:47.784306 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.783934 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56bfd54c55-jbczs" event={"ID":"38e48c60-7ea2-4d2d-9b68-0474e66ce230","Type":"ContainerDied","Data":"b4ea533d54ac08af4db95b0df7b8d28b31d9b8ac0537bd7b588fad6658b2887a"} Jul 02 13:49:47.784306 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.784016 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56bfd54c55-jbczs" event={"ID":"38e48c60-7ea2-4d2d-9b68-0474e66ce230","Type":"ContainerDied","Data":"0598231c9fe0388280a9844a90f69bb5de447ca3d4d54ded9501e749f2e6b889"} Jul 02 13:49:47.784306 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.784040 2561 scope.go:117] "RemoveContainer" containerID="b4ea533d54ac08af4db95b0df7b8d28b31d9b8ac0537bd7b588fad6658b2887a" Jul 02 13:49:47.793332 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.793314 2561 scope.go:117] "RemoveContainer" containerID="b4ea533d54ac08af4db95b0df7b8d28b31d9b8ac0537bd7b588fad6658b2887a" Jul 02 13:49:47.793663 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:49:47.793643 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4ea533d54ac08af4db95b0df7b8d28b31d9b8ac0537bd7b588fad6658b2887a\": container with ID starting with b4ea533d54ac08af4db95b0df7b8d28b31d9b8ac0537bd7b588fad6658b2887a not found: ID does not exist" containerID="b4ea533d54ac08af4db95b0df7b8d28b31d9b8ac0537bd7b588fad6658b2887a" Jul 02 13:49:47.793729 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.793679 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4ea533d54ac08af4db95b0df7b8d28b31d9b8ac0537bd7b588fad6658b2887a"} err="failed to get container status \"b4ea533d54ac08af4db95b0df7b8d28b31d9b8ac0537bd7b588fad6658b2887a\": rpc error: code = NotFound desc = could not find container \"b4ea533d54ac08af4db95b0df7b8d28b31d9b8ac0537bd7b588fad6658b2887a\": container with ID starting with b4ea533d54ac08af4db95b0df7b8d28b31d9b8ac0537bd7b588fad6658b2887a not found: ID does not exist" Jul 02 13:49:47.806718 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.806682 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56bfd54c55-jbczs"] Jul 02 13:49:47.810832 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:47.810806 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56bfd54c55-jbczs"] Jul 02 13:49:48.363674 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:49:48.363642 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38e48c60-7ea2-4d2d-9b68-0474e66ce230" path="/var/lib/kubelet/pods/38e48c60-7ea2-4d2d-9b68-0474e66ce230/volumes" Jul 02 13:50:01.404349 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:01.404313 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-x98tr"] Jul 02 13:50:01.404746 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:01.404700 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38e48c60-7ea2-4d2d-9b68-0474e66ce230" containerName="console" Jul 02 13:50:01.404746 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:01.404711 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="38e48c60-7ea2-4d2d-9b68-0474e66ce230" containerName="console" Jul 02 13:50:01.404819 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:01.404776 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="38e48c60-7ea2-4d2d-9b68-0474e66ce230" containerName="console" Jul 02 13:50:01.407654 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:01.407638 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x98tr" Jul 02 13:50:01.412399 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:01.412374 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 13:50:01.413038 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:01.413020 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 13:50:01.418442 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:01.418421 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-h7v2c\"" Jul 02 13:50:01.418572 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:01.418419 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 02 13:50:01.418572 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:01.418428 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 02 13:50:01.427883 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:01.427860 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-x98tr"] Jul 02 13:50:01.455395 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:01.455353 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnd48\" (UniqueName: \"kubernetes.io/projected/e50a83c3-13f2-407b-bb01-0944ac952a86-kube-api-access-qnd48\") pod \"kuadrant-console-plugin-66b5774985-x98tr\" (UID: \"e50a83c3-13f2-407b-bb01-0944ac952a86\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x98tr" Jul 02 13:50:01.455568 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:01.455414 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e50a83c3-13f2-407b-bb01-0944ac952a86-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-x98tr\" (UID: \"e50a83c3-13f2-407b-bb01-0944ac952a86\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x98tr" Jul 02 13:50:01.455568 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:01.455442 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e50a83c3-13f2-407b-bb01-0944ac952a86-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-x98tr\" (UID: \"e50a83c3-13f2-407b-bb01-0944ac952a86\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x98tr" Jul 02 13:50:01.556065 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:01.556025 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e50a83c3-13f2-407b-bb01-0944ac952a86-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-x98tr\" (UID: \"e50a83c3-13f2-407b-bb01-0944ac952a86\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x98tr" Jul 02 13:50:01.556226 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:01.556078 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e50a83c3-13f2-407b-bb01-0944ac952a86-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-x98tr\" (UID: \"e50a83c3-13f2-407b-bb01-0944ac952a86\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x98tr" Jul 02 13:50:01.556226 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:50:01.556102 2561 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 02 13:50:01.556226 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:50:01.556182 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e50a83c3-13f2-407b-bb01-0944ac952a86-plugin-serving-cert podName:e50a83c3-13f2-407b-bb01-0944ac952a86 nodeName:}" failed. No retries permitted until 2026-07-02 13:50:02.05616152 +0000 UTC m=+508.182298007 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/e50a83c3-13f2-407b-bb01-0944ac952a86-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-x98tr" (UID: "e50a83c3-13f2-407b-bb01-0944ac952a86") : secret "plugin-serving-cert" not found Jul 02 13:50:01.556360 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:01.556259 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnd48\" (UniqueName: \"kubernetes.io/projected/e50a83c3-13f2-407b-bb01-0944ac952a86-kube-api-access-qnd48\") pod \"kuadrant-console-plugin-66b5774985-x98tr\" (UID: \"e50a83c3-13f2-407b-bb01-0944ac952a86\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x98tr" Jul 02 13:50:01.556667 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:01.556648 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e50a83c3-13f2-407b-bb01-0944ac952a86-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-x98tr\" (UID: \"e50a83c3-13f2-407b-bb01-0944ac952a86\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x98tr" Jul 02 13:50:01.590606 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:01.590576 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnd48\" (UniqueName: \"kubernetes.io/projected/e50a83c3-13f2-407b-bb01-0944ac952a86-kube-api-access-qnd48\") pod \"kuadrant-console-plugin-66b5774985-x98tr\" (UID: \"e50a83c3-13f2-407b-bb01-0944ac952a86\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x98tr" Jul 02 13:50:02.061327 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:02.061282 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e50a83c3-13f2-407b-bb01-0944ac952a86-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-x98tr\" (UID: \"e50a83c3-13f2-407b-bb01-0944ac952a86\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x98tr" Jul 02 13:50:02.063678 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:02.063645 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/e50a83c3-13f2-407b-bb01-0944ac952a86-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-x98tr\" (UID: \"e50a83c3-13f2-407b-bb01-0944ac952a86\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x98tr" Jul 02 13:50:02.317013 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:02.316898 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x98tr" Jul 02 13:50:02.448802 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:02.448770 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-x98tr"] Jul 02 13:50:02.452632 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:50:02.452606 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode50a83c3_13f2_407b_bb01_0944ac952a86.slice/crio-026a46c875a42b45a4fbe9ba61d1b46209497ac6d875679e68c986374bb31c8d WatchSource:0}: Error finding container 026a46c875a42b45a4fbe9ba61d1b46209497ac6d875679e68c986374bb31c8d: Status 404 returned error can't find the container with id 026a46c875a42b45a4fbe9ba61d1b46209497ac6d875679e68c986374bb31c8d Jul 02 13:50:02.855544 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:02.855508 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x98tr" event={"ID":"e50a83c3-13f2-407b-bb01-0944ac952a86","Type":"ContainerStarted","Data":"026a46c875a42b45a4fbe9ba61d1b46209497ac6d875679e68c986374bb31c8d"} Jul 02 13:50:26.991146 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:26.991107 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x98tr" event={"ID":"e50a83c3-13f2-407b-bb01-0944ac952a86","Type":"ContainerStarted","Data":"ebfaa0fd063ec1944f51ffc0f7f84608448c546ebfcd5ebc3f8ca09967cf340a"} Jul 02 13:50:27.007820 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:27.007773 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-x98tr" podStartSLOduration=1.968204433 podStartE2EDuration="26.007759006s" totalImagesPullingTime="24.039554573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:50:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:50:27.006142567 +0000 UTC m=+533.132279075" watchObservedRunningTime="2026-07-02 13:50:27.007759006 +0000 UTC m=+533.133895511" Jul 02 13:50:43.704134 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:43.704099 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-m58rk"] Jul 02 13:50:43.736950 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:43.736913 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-m58rk"] Jul 02 13:50:43.737141 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:43.737042 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-m58rk" Jul 02 13:50:43.739582 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:43.739550 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 02 13:50:43.794954 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:43.794920 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-m58rk"] Jul 02 13:50:43.869434 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:43.869398 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnqnl\" (UniqueName: \"kubernetes.io/projected/00ab697b-dc97-4530-aeaf-791f0cbfc0bc-kube-api-access-lnqnl\") pod \"limitador-limitador-64c94fd64f-m58rk\" (UID: \"00ab697b-dc97-4530-aeaf-791f0cbfc0bc\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-m58rk" Jul 02 13:50:43.869607 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:43.869457 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/00ab697b-dc97-4530-aeaf-791f0cbfc0bc-config-file\") pod \"limitador-limitador-64c94fd64f-m58rk\" (UID: \"00ab697b-dc97-4530-aeaf-791f0cbfc0bc\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-m58rk" Jul 02 13:50:43.970553 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:43.970522 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/00ab697b-dc97-4530-aeaf-791f0cbfc0bc-config-file\") pod \"limitador-limitador-64c94fd64f-m58rk\" (UID: \"00ab697b-dc97-4530-aeaf-791f0cbfc0bc\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-m58rk" Jul 02 13:50:43.970759 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:43.970633 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnqnl\" (UniqueName: \"kubernetes.io/projected/00ab697b-dc97-4530-aeaf-791f0cbfc0bc-kube-api-access-lnqnl\") pod \"limitador-limitador-64c94fd64f-m58rk\" (UID: \"00ab697b-dc97-4530-aeaf-791f0cbfc0bc\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-m58rk" Jul 02 13:50:43.971180 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:43.971160 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/00ab697b-dc97-4530-aeaf-791f0cbfc0bc-config-file\") pod \"limitador-limitador-64c94fd64f-m58rk\" (UID: \"00ab697b-dc97-4530-aeaf-791f0cbfc0bc\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-m58rk" Jul 02 13:50:43.978655 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:43.978630 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnqnl\" (UniqueName: \"kubernetes.io/projected/00ab697b-dc97-4530-aeaf-791f0cbfc0bc-kube-api-access-lnqnl\") pod \"limitador-limitador-64c94fd64f-m58rk\" (UID: \"00ab697b-dc97-4530-aeaf-791f0cbfc0bc\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-m58rk" Jul 02 13:50:44.048086 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:44.048057 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-m58rk" Jul 02 13:50:44.180787 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:44.180745 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-m58rk"] Jul 02 13:50:44.183016 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:50:44.182988 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00ab697b_dc97_4530_aeaf_791f0cbfc0bc.slice/crio-04956bf5096f6e8d121b6e036ce823c74dfcd62b68db8efb30a81050da3e777d WatchSource:0}: Error finding container 04956bf5096f6e8d121b6e036ce823c74dfcd62b68db8efb30a81050da3e777d: Status 404 returned error can't find the container with id 04956bf5096f6e8d121b6e036ce823c74dfcd62b68db8efb30a81050da3e777d Jul 02 13:50:44.600382 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:44.600356 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-kpm9c"] Jul 02 13:50:44.604456 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:44.604439 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kpm9c" Jul 02 13:50:44.606417 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:44.606382 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-j96wk\"" Jul 02 13:50:44.610352 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:44.610331 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kpm9c"] Jul 02 13:50:44.677045 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:44.677008 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcd8x\" (UniqueName: \"kubernetes.io/projected/ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8-kube-api-access-mcd8x\") pod \"authorino-75d556bd77-kpm9c\" (UID: \"ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8\") " pod="kuadrant-system/authorino-75d556bd77-kpm9c" Jul 02 13:50:44.777910 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:44.777865 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcd8x\" (UniqueName: \"kubernetes.io/projected/ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8-kube-api-access-mcd8x\") pod \"authorino-75d556bd77-kpm9c\" (UID: \"ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8\") " pod="kuadrant-system/authorino-75d556bd77-kpm9c" Jul 02 13:50:44.786022 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:44.785987 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcd8x\" (UniqueName: \"kubernetes.io/projected/ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8-kube-api-access-mcd8x\") pod \"authorino-75d556bd77-kpm9c\" (UID: \"ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8\") " pod="kuadrant-system/authorino-75d556bd77-kpm9c" Jul 02 13:50:44.914956 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:44.914887 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kpm9c" Jul 02 13:50:45.064952 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:45.064882 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kpm9c"] Jul 02 13:50:45.069086 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:50:45.069047 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba4fe7a8_760a_4ef8_b5dd_b64e24bb29c8.slice/crio-4d5f678b375bec5158df297d48e43ff6a6ecaa49baf44a3d0ef15d0d5ee55922 WatchSource:0}: Error finding container 4d5f678b375bec5158df297d48e43ff6a6ecaa49baf44a3d0ef15d0d5ee55922: Status 404 returned error can't find the container with id 4d5f678b375bec5158df297d48e43ff6a6ecaa49baf44a3d0ef15d0d5ee55922 Jul 02 13:50:45.080539 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:45.080504 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kpm9c" event={"ID":"ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8","Type":"ContainerStarted","Data":"4d5f678b375bec5158df297d48e43ff6a6ecaa49baf44a3d0ef15d0d5ee55922"} Jul 02 13:50:45.082409 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:45.082334 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-m58rk" event={"ID":"00ab697b-dc97-4530-aeaf-791f0cbfc0bc","Type":"ContainerStarted","Data":"04956bf5096f6e8d121b6e036ce823c74dfcd62b68db8efb30a81050da3e777d"} Jul 02 13:50:49.106996 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:49.106949 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kpm9c" event={"ID":"ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8","Type":"ContainerStarted","Data":"f545d00c8bdb86f521448f38d4169033de46263d4624acf8285ddbbfe4e6e2d3"} Jul 02 13:50:49.108738 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:49.108709 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-m58rk" event={"ID":"00ab697b-dc97-4530-aeaf-791f0cbfc0bc","Type":"ContainerStarted","Data":"1b34216935718e1bee581679d966086a9a96e9e3361a7d9ba1c9fc493df02093"} Jul 02 13:50:49.108868 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:49.108767 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-m58rk" Jul 02 13:50:49.121238 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:49.121184 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-kpm9c" podStartSLOduration=1.171297874 podStartE2EDuration="5.12116489s" totalImagesPullingTime="3.949867016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:50:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:50:49.120702906 +0000 UTC m=+555.246839411" watchObservedRunningTime="2026-07-02 13:50:49.12116489 +0000 UTC m=+555.247301395" Jul 02 13:50:49.136424 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:49.136351 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-m58rk" podStartSLOduration=1.357823185 podStartE2EDuration="6.136328141s" totalImagesPullingTime="4.778504956s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:50:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:50:49.135521149 +0000 UTC m=+555.261657667" watchObservedRunningTime="2026-07-02 13:50:49.136328141 +0000 UTC m=+555.262464647" Jul 02 13:50:59.202612 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:59.202572 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-m58rk"] Jul 02 13:50:59.203199 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:59.202751 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-m58rk" podUID="00ab697b-dc97-4530-aeaf-791f0cbfc0bc" containerName="limitador" containerID="cri-o://1b34216935718e1bee581679d966086a9a96e9e3361a7d9ba1c9fc493df02093" gracePeriod=30 Jul 02 13:50:59.203568 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:59.203444 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-m58rk" Jul 02 13:50:59.750536 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:59.750507 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-m58rk" Jul 02 13:50:59.927904 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:59.927811 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/00ab697b-dc97-4530-aeaf-791f0cbfc0bc-config-file\") pod \"00ab697b-dc97-4530-aeaf-791f0cbfc0bc\" (UID: \"00ab697b-dc97-4530-aeaf-791f0cbfc0bc\") " Jul 02 13:50:59.928096 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:59.927915 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnqnl\" (UniqueName: \"kubernetes.io/projected/00ab697b-dc97-4530-aeaf-791f0cbfc0bc-kube-api-access-lnqnl\") pod \"00ab697b-dc97-4530-aeaf-791f0cbfc0bc\" (UID: \"00ab697b-dc97-4530-aeaf-791f0cbfc0bc\") " Jul 02 13:50:59.928272 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:59.928252 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00ab697b-dc97-4530-aeaf-791f0cbfc0bc-config-file" (OuterVolumeSpecName: "config-file") pod "00ab697b-dc97-4530-aeaf-791f0cbfc0bc" (UID: "00ab697b-dc97-4530-aeaf-791f0cbfc0bc"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 13:50:59.930108 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:50:59.930081 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00ab697b-dc97-4530-aeaf-791f0cbfc0bc-kube-api-access-lnqnl" (OuterVolumeSpecName: "kube-api-access-lnqnl") pod "00ab697b-dc97-4530-aeaf-791f0cbfc0bc" (UID: "00ab697b-dc97-4530-aeaf-791f0cbfc0bc"). InnerVolumeSpecName "kube-api-access-lnqnl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:51:00.029328 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:00.029284 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnqnl\" (UniqueName: \"kubernetes.io/projected/00ab697b-dc97-4530-aeaf-791f0cbfc0bc-kube-api-access-lnqnl\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:51:00.029328 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:00.029321 2561 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/00ab697b-dc97-4530-aeaf-791f0cbfc0bc-config-file\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:51:00.164253 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:00.164214 2561 generic.go:356] "Generic (PLEG): container finished" podID="00ab697b-dc97-4530-aeaf-791f0cbfc0bc" containerID="1b34216935718e1bee581679d966086a9a96e9e3361a7d9ba1c9fc493df02093" exitCode=0 Jul 02 13:51:00.164448 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:00.164261 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-m58rk" Jul 02 13:51:00.164448 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:00.164279 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-m58rk" event={"ID":"00ab697b-dc97-4530-aeaf-791f0cbfc0bc","Type":"ContainerDied","Data":"1b34216935718e1bee581679d966086a9a96e9e3361a7d9ba1c9fc493df02093"} Jul 02 13:51:00.164448 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:00.164355 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-m58rk" event={"ID":"00ab697b-dc97-4530-aeaf-791f0cbfc0bc","Type":"ContainerDied","Data":"04956bf5096f6e8d121b6e036ce823c74dfcd62b68db8efb30a81050da3e777d"} Jul 02 13:51:00.164448 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:00.164379 2561 scope.go:117] "RemoveContainer" containerID="1b34216935718e1bee581679d966086a9a96e9e3361a7d9ba1c9fc493df02093" Jul 02 13:51:00.172561 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:00.172542 2561 scope.go:117] "RemoveContainer" containerID="1b34216935718e1bee581679d966086a9a96e9e3361a7d9ba1c9fc493df02093" Jul 02 13:51:00.172816 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:51:00.172794 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b34216935718e1bee581679d966086a9a96e9e3361a7d9ba1c9fc493df02093\": container with ID starting with 1b34216935718e1bee581679d966086a9a96e9e3361a7d9ba1c9fc493df02093 not found: ID does not exist" containerID="1b34216935718e1bee581679d966086a9a96e9e3361a7d9ba1c9fc493df02093" Jul 02 13:51:00.172913 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:00.172827 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b34216935718e1bee581679d966086a9a96e9e3361a7d9ba1c9fc493df02093"} err="failed to get container status \"1b34216935718e1bee581679d966086a9a96e9e3361a7d9ba1c9fc493df02093\": rpc error: code = NotFound desc = could not find container \"1b34216935718e1bee581679d966086a9a96e9e3361a7d9ba1c9fc493df02093\": container with ID starting with 1b34216935718e1bee581679d966086a9a96e9e3361a7d9ba1c9fc493df02093 not found: ID does not exist" Jul 02 13:51:00.186501 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:00.186330 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-m58rk"] Jul 02 13:51:00.188040 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:00.188017 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-m58rk"] Jul 02 13:51:00.363706 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:00.363660 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00ab697b-dc97-4530-aeaf-791f0cbfc0bc" path="/var/lib/kubelet/pods/00ab697b-dc97-4530-aeaf-791f0cbfc0bc/volumes" Jul 02 13:51:05.287746 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:05.287712 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-88cc6"] Jul 02 13:51:05.288250 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:05.288233 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00ab697b-dc97-4530-aeaf-791f0cbfc0bc" containerName="limitador" Jul 02 13:51:05.288298 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:05.288252 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="00ab697b-dc97-4530-aeaf-791f0cbfc0bc" containerName="limitador" Jul 02 13:51:05.288330 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:05.288310 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="00ab697b-dc97-4530-aeaf-791f0cbfc0bc" containerName="limitador" Jul 02 13:51:05.292664 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:05.292644 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-88cc6" Jul 02 13:51:05.296146 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:05.296121 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 02 13:51:05.298618 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:05.298597 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-5d2vt\"" Jul 02 13:51:05.370171 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:05.370139 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-88cc6"] Jul 02 13:51:05.476677 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:05.476639 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l857\" (UniqueName: \"kubernetes.io/projected/f496bfe6-dbd4-405c-90de-7239d985a474-kube-api-access-4l857\") pod \"postgres-69fb4594cf-88cc6\" (UID: \"f496bfe6-dbd4-405c-90de-7239d985a474\") " pod="opendatahub/postgres-69fb4594cf-88cc6" Jul 02 13:51:05.476854 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:05.476703 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f496bfe6-dbd4-405c-90de-7239d985a474-data\") pod \"postgres-69fb4594cf-88cc6\" (UID: \"f496bfe6-dbd4-405c-90de-7239d985a474\") " pod="opendatahub/postgres-69fb4594cf-88cc6" Jul 02 13:51:05.578242 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:05.578162 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l857\" (UniqueName: \"kubernetes.io/projected/f496bfe6-dbd4-405c-90de-7239d985a474-kube-api-access-4l857\") pod \"postgres-69fb4594cf-88cc6\" (UID: \"f496bfe6-dbd4-405c-90de-7239d985a474\") " pod="opendatahub/postgres-69fb4594cf-88cc6" Jul 02 13:51:05.578242 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:05.578206 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f496bfe6-dbd4-405c-90de-7239d985a474-data\") pod \"postgres-69fb4594cf-88cc6\" (UID: \"f496bfe6-dbd4-405c-90de-7239d985a474\") " pod="opendatahub/postgres-69fb4594cf-88cc6" Jul 02 13:51:05.578514 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:05.578498 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f496bfe6-dbd4-405c-90de-7239d985a474-data\") pod \"postgres-69fb4594cf-88cc6\" (UID: \"f496bfe6-dbd4-405c-90de-7239d985a474\") " pod="opendatahub/postgres-69fb4594cf-88cc6" Jul 02 13:51:05.587283 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:05.587251 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l857\" (UniqueName: \"kubernetes.io/projected/f496bfe6-dbd4-405c-90de-7239d985a474-kube-api-access-4l857\") pod \"postgres-69fb4594cf-88cc6\" (UID: \"f496bfe6-dbd4-405c-90de-7239d985a474\") " pod="opendatahub/postgres-69fb4594cf-88cc6" Jul 02 13:51:05.604732 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:05.604696 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-88cc6" Jul 02 13:51:05.739101 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:05.739072 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-88cc6"] Jul 02 13:51:05.741597 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:51:05.741571 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf496bfe6_dbd4_405c_90de_7239d985a474.slice/crio-b06322106a8c390dcae72ff03902fbb8eea02b0c00162d0c15adf4d2ec147a67 WatchSource:0}: Error finding container b06322106a8c390dcae72ff03902fbb8eea02b0c00162d0c15adf4d2ec147a67: Status 404 returned error can't find the container with id b06322106a8c390dcae72ff03902fbb8eea02b0c00162d0c15adf4d2ec147a67 Jul 02 13:51:06.195258 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:06.195221 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-88cc6" event={"ID":"f496bfe6-dbd4-405c-90de-7239d985a474","Type":"ContainerStarted","Data":"b06322106a8c390dcae72ff03902fbb8eea02b0c00162d0c15adf4d2ec147a67"} Jul 02 13:51:11.222167 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:11.222132 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-88cc6" event={"ID":"f496bfe6-dbd4-405c-90de-7239d985a474","Type":"ContainerStarted","Data":"0be746d6ef577dd0cee3a892c3431a167ed27bd20ce108be05bea2ecfd38a3aa"} Jul 02 13:51:11.222589 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:11.222312 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-88cc6" Jul 02 13:51:11.239051 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:11.239002 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-88cc6" podStartSLOduration=1.53960195 podStartE2EDuration="6.238985715s" totalImagesPullingTime="4.699383765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:51:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:51:11.237558092 +0000 UTC m=+577.363694596" watchObservedRunningTime="2026-07-02 13:51:11.238985715 +0000 UTC m=+577.365122212" Jul 02 13:51:17.257665 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:17.257629 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-88cc6" Jul 02 13:51:47.367131 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:47.367089 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 02 13:51:47.388853 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:47.388819 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 02 13:51:47.389062 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:47.388931 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 02 13:51:47.391496 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:47.391463 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jul 02 13:51:47.391627 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:47.391464 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jul 02 13:51:47.391627 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:47.391584 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jul 02 13:51:47.392277 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:47.392136 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-tj4wh\"" Jul 02 13:51:47.470353 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:47.470308 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kxv6\" (UniqueName: \"kubernetes.io/projected/76a4dbe8-f8a0-4e05-b358-82d17125c254-kube-api-access-2kxv6\") pod \"maas-keycloak-0\" (UID: \"76a4dbe8-f8a0-4e05-b358-82d17125c254\") " pod="keycloak-system/maas-keycloak-0" Jul 02 13:51:47.571708 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:47.571666 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kxv6\" (UniqueName: \"kubernetes.io/projected/76a4dbe8-f8a0-4e05-b358-82d17125c254-kube-api-access-2kxv6\") pod \"maas-keycloak-0\" (UID: \"76a4dbe8-f8a0-4e05-b358-82d17125c254\") " pod="keycloak-system/maas-keycloak-0" Jul 02 13:51:47.580888 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:47.580854 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kxv6\" (UniqueName: \"kubernetes.io/projected/76a4dbe8-f8a0-4e05-b358-82d17125c254-kube-api-access-2kxv6\") pod \"maas-keycloak-0\" (UID: \"76a4dbe8-f8a0-4e05-b358-82d17125c254\") " pod="keycloak-system/maas-keycloak-0" Jul 02 13:51:47.699724 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:47.699626 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 02 13:51:47.850428 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:47.850253 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 02 13:51:47.853390 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:51:47.853359 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76a4dbe8_f8a0_4e05_b358_82d17125c254.slice/crio-70cf81013b9267f288f350a8c5df38e3c9c80d30369ca2507db0976dd4e84380 WatchSource:0}: Error finding container 70cf81013b9267f288f350a8c5df38e3c9c80d30369ca2507db0976dd4e84380: Status 404 returned error can't find the container with id 70cf81013b9267f288f350a8c5df38e3c9c80d30369ca2507db0976dd4e84380 Jul 02 13:51:48.401236 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:48.401203 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"76a4dbe8-f8a0-4e05-b358-82d17125c254","Type":"ContainerStarted","Data":"70cf81013b9267f288f350a8c5df38e3c9c80d30369ca2507db0976dd4e84380"} Jul 02 13:51:54.440829 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:54.440780 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"76a4dbe8-f8a0-4e05-b358-82d17125c254","Type":"ContainerStarted","Data":"d48665fb46ebe70c337dbc1d47072dada6eeab4d28da3b26183115f20bbcbbed"} Jul 02 13:51:54.460762 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:54.460696 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.850142415 podStartE2EDuration="7.460675504s" totalImagesPullingTime="5.610533089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:51:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:51:54.460070391 +0000 UTC m=+620.586206896" watchObservedRunningTime="2026-07-02 13:51:54.460675504 +0000 UTC m=+620.586812010" Jul 02 13:51:54.700774 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:54.700668 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jul 02 13:51:54.702401 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:54.702349 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76a4dbe8-f8a0-4e05-b358-82d17125c254" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.36:9000/health/started\": dial tcp 10.134.0.36:9000: connect: connection refused" Jul 02 13:51:55.700834 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:55.700724 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76a4dbe8-f8a0-4e05-b358-82d17125c254" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.36:9000/health/started\": dial tcp 10.134.0.36:9000: connect: connection refused" Jul 02 13:51:56.700773 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:56.700723 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76a4dbe8-f8a0-4e05-b358-82d17125c254" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.36:9000/health/started\": dial tcp 10.134.0.36:9000: connect: connection refused" Jul 02 13:51:57.700922 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:57.700224 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jul 02 13:51:57.700922 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:57.700513 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76a4dbe8-f8a0-4e05-b358-82d17125c254" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.36:9000/health/started\": dial tcp 10.134.0.36:9000: connect: connection refused" Jul 02 13:51:58.700240 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:58.700182 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76a4dbe8-f8a0-4e05-b358-82d17125c254" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.36:9000/health/started\": dial tcp 10.134.0.36:9000: connect: connection refused" Jul 02 13:51:59.701058 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:51:59.701008 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76a4dbe8-f8a0-4e05-b358-82d17125c254" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.36:9000/health/started\": dial tcp 10.134.0.36:9000: connect: connection refused" Jul 02 13:52:00.701583 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:00.701540 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76a4dbe8-f8a0-4e05-b358-82d17125c254" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.36:9000/health/started\": dial tcp 10.134.0.36:9000: connect: connection refused" Jul 02 13:52:01.700258 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:01.700210 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76a4dbe8-f8a0-4e05-b358-82d17125c254" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.36:9000/health/started\": dial tcp 10.134.0.36:9000: connect: connection refused" Jul 02 13:52:02.700699 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:02.700640 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76a4dbe8-f8a0-4e05-b358-82d17125c254" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.36:9000/health/started\": dial tcp 10.134.0.36:9000: connect: connection refused" Jul 02 13:52:03.700467 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:03.700423 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76a4dbe8-f8a0-4e05-b358-82d17125c254" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.36:9000/health/started\": dial tcp 10.134.0.36:9000: connect: connection refused" Jul 02 13:52:04.700313 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:04.700263 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76a4dbe8-f8a0-4e05-b358-82d17125c254" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.36:9000/health/started\": dial tcp 10.134.0.36:9000: connect: connection refused" Jul 02 13:52:05.700352 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:05.700303 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76a4dbe8-f8a0-4e05-b358-82d17125c254" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.36:9000/health/started\": dial tcp 10.134.0.36:9000: connect: connection refused" Jul 02 13:52:06.700232 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:06.700180 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="76a4dbe8-f8a0-4e05-b358-82d17125c254" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.36:9000/health/started\": dial tcp 10.134.0.36:9000: connect: connection refused" Jul 02 13:52:07.830082 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:07.830034 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jul 02 13:52:07.852274 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:07.852221 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="76a4dbe8-f8a0-4e05-b358-82d17125c254" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 13:52:17.836241 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:17.836196 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jul 02 13:52:20.705763 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:20.705730 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kpm9c"] Jul 02 13:52:20.706209 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:20.705901 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-kpm9c" podUID="ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8" containerName="authorino" containerID="cri-o://f545d00c8bdb86f521448f38d4169033de46263d4624acf8285ddbbfe4e6e2d3" gracePeriod=30 Jul 02 13:52:20.959415 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:20.959395 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kpm9c" Jul 02 13:52:21.010632 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:21.010596 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mcd8x\" (UniqueName: \"kubernetes.io/projected/ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8-kube-api-access-mcd8x\") pod \"ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8\" (UID: \"ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8\") " Jul 02 13:52:21.012684 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:21.012645 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8-kube-api-access-mcd8x" (OuterVolumeSpecName: "kube-api-access-mcd8x") pod "ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8" (UID: "ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8"). InnerVolumeSpecName "kube-api-access-mcd8x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:52:21.111476 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:21.111435 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcd8x\" (UniqueName: \"kubernetes.io/projected/ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8-kube-api-access-mcd8x\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:52:21.611961 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:21.611920 2561 generic.go:356] "Generic (PLEG): container finished" podID="ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8" containerID="f545d00c8bdb86f521448f38d4169033de46263d4624acf8285ddbbfe4e6e2d3" exitCode=0 Jul 02 13:52:21.612165 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:21.611964 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kpm9c" Jul 02 13:52:21.612165 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:21.612013 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kpm9c" event={"ID":"ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8","Type":"ContainerDied","Data":"f545d00c8bdb86f521448f38d4169033de46263d4624acf8285ddbbfe4e6e2d3"} Jul 02 13:52:21.612165 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:21.612062 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kpm9c" event={"ID":"ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8","Type":"ContainerDied","Data":"4d5f678b375bec5158df297d48e43ff6a6ecaa49baf44a3d0ef15d0d5ee55922"} Jul 02 13:52:21.612165 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:21.612081 2561 scope.go:117] "RemoveContainer" containerID="f545d00c8bdb86f521448f38d4169033de46263d4624acf8285ddbbfe4e6e2d3" Jul 02 13:52:21.621400 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:21.621380 2561 scope.go:117] "RemoveContainer" containerID="f545d00c8bdb86f521448f38d4169033de46263d4624acf8285ddbbfe4e6e2d3" Jul 02 13:52:21.621712 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:52:21.621686 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f545d00c8bdb86f521448f38d4169033de46263d4624acf8285ddbbfe4e6e2d3\": container with ID starting with f545d00c8bdb86f521448f38d4169033de46263d4624acf8285ddbbfe4e6e2d3 not found: ID does not exist" containerID="f545d00c8bdb86f521448f38d4169033de46263d4624acf8285ddbbfe4e6e2d3" Jul 02 13:52:21.621798 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:21.621722 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f545d00c8bdb86f521448f38d4169033de46263d4624acf8285ddbbfe4e6e2d3"} err="failed to get container status \"f545d00c8bdb86f521448f38d4169033de46263d4624acf8285ddbbfe4e6e2d3\": rpc error: code = NotFound desc = could not find container \"f545d00c8bdb86f521448f38d4169033de46263d4624acf8285ddbbfe4e6e2d3\": container with ID starting with f545d00c8bdb86f521448f38d4169033de46263d4624acf8285ddbbfe4e6e2d3 not found: ID does not exist" Jul 02 13:52:21.633988 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:21.633944 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kpm9c"] Jul 02 13:52:21.637490 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:21.637460 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kpm9c"] Jul 02 13:52:22.364048 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:22.364012 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8" path="/var/lib/kubelet/pods/ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8/volumes" Jul 02 13:52:25.105203 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.105167 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c4db889fc-5xwz2"] Jul 02 13:52:25.105606 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.105571 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8" containerName="authorino" Jul 02 13:52:25.105606 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.105582 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8" containerName="authorino" Jul 02 13:52:25.105683 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.105665 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba4fe7a8-760a-4ef8-b5dd-b64e24bb29c8" containerName="authorino" Jul 02 13:52:25.187844 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.187809 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c4db889fc-5xwz2"] Jul 02 13:52:25.188066 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.187905 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c4db889fc-5xwz2" Jul 02 13:52:25.191620 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.191595 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-pt644\"" Jul 02 13:52:25.191741 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.191597 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 02 13:52:25.191741 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.191688 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 02 13:52:25.249835 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.249801 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8g94\" (UniqueName: \"kubernetes.io/projected/b190f8b6-8c4b-4768-b268-e55b0231c329-kube-api-access-l8g94\") pod \"maas-controller-6c4db889fc-5xwz2\" (UID: \"b190f8b6-8c4b-4768-b268-e55b0231c329\") " pod="opendatahub/maas-controller-6c4db889fc-5xwz2" Jul 02 13:52:25.249835 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.249837 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b190f8b6-8c4b-4768-b268-e55b0231c329-webhook-cert\") pod \"maas-controller-6c4db889fc-5xwz2\" (UID: \"b190f8b6-8c4b-4768-b268-e55b0231c329\") " pod="opendatahub/maas-controller-6c4db889fc-5xwz2" Jul 02 13:52:25.315791 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.315755 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56c678849-slc8b"] Jul 02 13:52:25.340151 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.340117 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56c678849-slc8b"] Jul 02 13:52:25.340317 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.340220 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56c678849-slc8b" Jul 02 13:52:25.350628 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.350605 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8g94\" (UniqueName: \"kubernetes.io/projected/b190f8b6-8c4b-4768-b268-e55b0231c329-kube-api-access-l8g94\") pod \"maas-controller-6c4db889fc-5xwz2\" (UID: \"b190f8b6-8c4b-4768-b268-e55b0231c329\") " pod="opendatahub/maas-controller-6c4db889fc-5xwz2" Jul 02 13:52:25.350780 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.350640 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b190f8b6-8c4b-4768-b268-e55b0231c329-webhook-cert\") pod \"maas-controller-6c4db889fc-5xwz2\" (UID: \"b190f8b6-8c4b-4768-b268-e55b0231c329\") " pod="opendatahub/maas-controller-6c4db889fc-5xwz2" Jul 02 13:52:25.353059 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.353037 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b190f8b6-8c4b-4768-b268-e55b0231c329-webhook-cert\") pod \"maas-controller-6c4db889fc-5xwz2\" (UID: \"b190f8b6-8c4b-4768-b268-e55b0231c329\") " pod="opendatahub/maas-controller-6c4db889fc-5xwz2" Jul 02 13:52:25.362396 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.362345 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8g94\" (UniqueName: \"kubernetes.io/projected/b190f8b6-8c4b-4768-b268-e55b0231c329-kube-api-access-l8g94\") pod \"maas-controller-6c4db889fc-5xwz2\" (UID: \"b190f8b6-8c4b-4768-b268-e55b0231c329\") " pod="opendatahub/maas-controller-6c4db889fc-5xwz2" Jul 02 13:52:25.451323 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.451288 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d72ws\" (UniqueName: \"kubernetes.io/projected/ed8b896e-29aa-4383-ae97-57911e4c99ff-kube-api-access-d72ws\") pod \"maas-controller-56c678849-slc8b\" (UID: \"ed8b896e-29aa-4383-ae97-57911e4c99ff\") " pod="opendatahub/maas-controller-56c678849-slc8b" Jul 02 13:52:25.451486 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.451376 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed8b896e-29aa-4383-ae97-57911e4c99ff-webhook-cert\") pod \"maas-controller-56c678849-slc8b\" (UID: \"ed8b896e-29aa-4383-ae97-57911e4c99ff\") " pod="opendatahub/maas-controller-56c678849-slc8b" Jul 02 13:52:25.498218 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.498179 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c4db889fc-5xwz2" Jul 02 13:52:25.552349 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.552299 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d72ws\" (UniqueName: \"kubernetes.io/projected/ed8b896e-29aa-4383-ae97-57911e4c99ff-kube-api-access-d72ws\") pod \"maas-controller-56c678849-slc8b\" (UID: \"ed8b896e-29aa-4383-ae97-57911e4c99ff\") " pod="opendatahub/maas-controller-56c678849-slc8b" Jul 02 13:52:25.552556 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.552403 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed8b896e-29aa-4383-ae97-57911e4c99ff-webhook-cert\") pod \"maas-controller-56c678849-slc8b\" (UID: \"ed8b896e-29aa-4383-ae97-57911e4c99ff\") " pod="opendatahub/maas-controller-56c678849-slc8b" Jul 02 13:52:25.555216 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.555171 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ed8b896e-29aa-4383-ae97-57911e4c99ff-webhook-cert\") pod \"maas-controller-56c678849-slc8b\" (UID: \"ed8b896e-29aa-4383-ae97-57911e4c99ff\") " pod="opendatahub/maas-controller-56c678849-slc8b" Jul 02 13:52:25.561877 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.561786 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d72ws\" (UniqueName: \"kubernetes.io/projected/ed8b896e-29aa-4383-ae97-57911e4c99ff-kube-api-access-d72ws\") pod \"maas-controller-56c678849-slc8b\" (UID: \"ed8b896e-29aa-4383-ae97-57911e4c99ff\") " pod="opendatahub/maas-controller-56c678849-slc8b" Jul 02 13:52:25.651007 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.650905 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56c678849-slc8b" Jul 02 13:52:25.663203 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.662322 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c4db889fc-5xwz2"] Jul 02 13:52:25.666293 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.666270 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 13:52:25.788536 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:25.788495 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56c678849-slc8b"] Jul 02 13:52:25.791204 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:52:25.791169 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded8b896e_29aa_4383_ae97_57911e4c99ff.slice/crio-095430133c1309903e19071202462ef9f8350ce73abecb0137e48bdc54d924d6 WatchSource:0}: Error finding container 095430133c1309903e19071202462ef9f8350ce73abecb0137e48bdc54d924d6: Status 404 returned error can't find the container with id 095430133c1309903e19071202462ef9f8350ce73abecb0137e48bdc54d924d6 Jul 02 13:52:26.643466 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:26.643377 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c4db889fc-5xwz2" event={"ID":"b190f8b6-8c4b-4768-b268-e55b0231c329","Type":"ContainerStarted","Data":"3e93a9a4c3c49c13e465a2ede52feda4ff24e9d582de0ab006e35cb97f1ecf67"} Jul 02 13:52:26.645799 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:26.645764 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56c678849-slc8b" event={"ID":"ed8b896e-29aa-4383-ae97-57911e4c99ff","Type":"ContainerStarted","Data":"095430133c1309903e19071202462ef9f8350ce73abecb0137e48bdc54d924d6"} Jul 02 13:52:29.663533 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:29.663479 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56c678849-slc8b" event={"ID":"ed8b896e-29aa-4383-ae97-57911e4c99ff","Type":"ContainerStarted","Data":"7205209c7a4b096e60d75b94e8cdc54341305b9abf1f80e6e607e94675d3f01f"} Jul 02 13:52:29.664006 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:29.663552 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56c678849-slc8b" Jul 02 13:52:29.665090 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:29.665053 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c4db889fc-5xwz2" event={"ID":"b190f8b6-8c4b-4768-b268-e55b0231c329","Type":"ContainerStarted","Data":"8ed811fad73133bd8e7b904a79306818d3eca417f1c173dda00fc9ee1791af9a"} Jul 02 13:52:29.665209 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:29.665125 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c4db889fc-5xwz2" Jul 02 13:52:29.684737 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:29.684668 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56c678849-slc8b" podStartSLOduration=1.835670436 podStartE2EDuration="4.684646495s" totalImagesPullingTime="2.848976059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:52:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:52:29.683821801 +0000 UTC m=+655.809958308" watchObservedRunningTime="2026-07-02 13:52:29.684646495 +0000 UTC m=+655.810783002" Jul 02 13:52:29.706316 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:29.706258 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c4db889fc-5xwz2" podStartSLOduration=1.740378052 podStartE2EDuration="4.706240307s" totalImagesPullingTime="2.965862255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:52:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:52:29.705291704 +0000 UTC m=+655.831428228" watchObservedRunningTime="2026-07-02 13:52:29.706240307 +0000 UTC m=+655.832376812" Jul 02 13:52:40.688031 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:40.687994 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56c678849-slc8b" Jul 02 13:52:40.688429 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:40.688078 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c4db889fc-5xwz2" Jul 02 13:52:40.782357 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:40.782308 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c4db889fc-5xwz2"] Jul 02 13:52:40.782605 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:40.782560 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6c4db889fc-5xwz2" podUID="b190f8b6-8c4b-4768-b268-e55b0231c329" containerName="manager" containerID="cri-o://8ed811fad73133bd8e7b904a79306818d3eca417f1c173dda00fc9ee1791af9a" gracePeriod=30 Jul 02 13:52:41.033611 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:41.033587 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c4db889fc-5xwz2" Jul 02 13:52:41.102914 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:41.102876 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8g94\" (UniqueName: \"kubernetes.io/projected/b190f8b6-8c4b-4768-b268-e55b0231c329-kube-api-access-l8g94\") pod \"b190f8b6-8c4b-4768-b268-e55b0231c329\" (UID: \"b190f8b6-8c4b-4768-b268-e55b0231c329\") " Jul 02 13:52:41.102914 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:41.102913 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b190f8b6-8c4b-4768-b268-e55b0231c329-webhook-cert\") pod \"b190f8b6-8c4b-4768-b268-e55b0231c329\" (UID: \"b190f8b6-8c4b-4768-b268-e55b0231c329\") " Jul 02 13:52:41.105195 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:41.105165 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b190f8b6-8c4b-4768-b268-e55b0231c329-kube-api-access-l8g94" (OuterVolumeSpecName: "kube-api-access-l8g94") pod "b190f8b6-8c4b-4768-b268-e55b0231c329" (UID: "b190f8b6-8c4b-4768-b268-e55b0231c329"). InnerVolumeSpecName "kube-api-access-l8g94". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:52:41.105319 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:41.105210 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b190f8b6-8c4b-4768-b268-e55b0231c329-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b190f8b6-8c4b-4768-b268-e55b0231c329" (UID: "b190f8b6-8c4b-4768-b268-e55b0231c329"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 13:52:41.204074 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:41.203949 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8g94\" (UniqueName: \"kubernetes.io/projected/b190f8b6-8c4b-4768-b268-e55b0231c329-kube-api-access-l8g94\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:52:41.204074 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:41.204021 2561 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b190f8b6-8c4b-4768-b268-e55b0231c329-webhook-cert\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:52:41.727146 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:41.727109 2561 generic.go:356] "Generic (PLEG): container finished" podID="b190f8b6-8c4b-4768-b268-e55b0231c329" containerID="8ed811fad73133bd8e7b904a79306818d3eca417f1c173dda00fc9ee1791af9a" exitCode=0 Jul 02 13:52:41.727571 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:41.727152 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c4db889fc-5xwz2" Jul 02 13:52:41.727571 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:41.727190 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c4db889fc-5xwz2" event={"ID":"b190f8b6-8c4b-4768-b268-e55b0231c329","Type":"ContainerDied","Data":"8ed811fad73133bd8e7b904a79306818d3eca417f1c173dda00fc9ee1791af9a"} Jul 02 13:52:41.727571 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:41.727249 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c4db889fc-5xwz2" event={"ID":"b190f8b6-8c4b-4768-b268-e55b0231c329","Type":"ContainerDied","Data":"3e93a9a4c3c49c13e465a2ede52feda4ff24e9d582de0ab006e35cb97f1ecf67"} Jul 02 13:52:41.727571 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:41.727277 2561 scope.go:117] "RemoveContainer" containerID="8ed811fad73133bd8e7b904a79306818d3eca417f1c173dda00fc9ee1791af9a" Jul 02 13:52:41.736244 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:41.736225 2561 scope.go:117] "RemoveContainer" containerID="8ed811fad73133bd8e7b904a79306818d3eca417f1c173dda00fc9ee1791af9a" Jul 02 13:52:41.736555 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:52:41.736535 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ed811fad73133bd8e7b904a79306818d3eca417f1c173dda00fc9ee1791af9a\": container with ID starting with 8ed811fad73133bd8e7b904a79306818d3eca417f1c173dda00fc9ee1791af9a not found: ID does not exist" containerID="8ed811fad73133bd8e7b904a79306818d3eca417f1c173dda00fc9ee1791af9a" Jul 02 13:52:41.736612 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:41.736564 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ed811fad73133bd8e7b904a79306818d3eca417f1c173dda00fc9ee1791af9a"} err="failed to get container status \"8ed811fad73133bd8e7b904a79306818d3eca417f1c173dda00fc9ee1791af9a\": rpc error: code = NotFound desc = could not find container \"8ed811fad73133bd8e7b904a79306818d3eca417f1c173dda00fc9ee1791af9a\": container with ID starting with 8ed811fad73133bd8e7b904a79306818d3eca417f1c173dda00fc9ee1791af9a not found: ID does not exist" Jul 02 13:52:41.752339 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:41.752308 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c4db889fc-5xwz2"] Jul 02 13:52:41.771087 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:41.765808 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c4db889fc-5xwz2"] Jul 02 13:52:42.363761 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:52:42.363727 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b190f8b6-8c4b-4768-b268-e55b0231c329" path="/var/lib/kubelet/pods/b190f8b6-8c4b-4768-b268-e55b0231c329/volumes" Jul 02 13:53:00.998442 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:00.998406 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-86f58868c9-8jw5w"] Jul 02 13:53:00.998831 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:00.998803 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b190f8b6-8c4b-4768-b268-e55b0231c329" containerName="manager" Jul 02 13:53:00.998831 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:00.998815 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="b190f8b6-8c4b-4768-b268-e55b0231c329" containerName="manager" Jul 02 13:53:00.998903 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:00.998877 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="b190f8b6-8c4b-4768-b268-e55b0231c329" containerName="manager" Jul 02 13:53:01.001959 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.001938 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-86f58868c9-8jw5w" Jul 02 13:53:01.004811 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.004778 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 02 13:53:01.004811 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.004788 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-gnsxp\"" Jul 02 13:53:01.011597 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.011556 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-86f58868c9-8jw5w"] Jul 02 13:53:01.083155 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.079519 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb695f8e-21bb-4604-a6fa-61e0a1b32150-openshift-service-ca\") pod \"maas-api-86f58868c9-8jw5w\" (UID: \"eb695f8e-21bb-4604-a6fa-61e0a1b32150\") " pod="opendatahub/maas-api-86f58868c9-8jw5w" Jul 02 13:53:01.083155 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.079581 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb695f8e-21bb-4604-a6fa-61e0a1b32150-maas-api-tls\") pod \"maas-api-86f58868c9-8jw5w\" (UID: \"eb695f8e-21bb-4604-a6fa-61e0a1b32150\") " pod="opendatahub/maas-api-86f58868c9-8jw5w" Jul 02 13:53:01.180292 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.180252 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws5jr\" (UniqueName: \"kubernetes.io/projected/eb695f8e-21bb-4604-a6fa-61e0a1b32150-kube-api-access-ws5jr\") pod \"maas-api-86f58868c9-8jw5w\" (UID: \"eb695f8e-21bb-4604-a6fa-61e0a1b32150\") " pod="opendatahub/maas-api-86f58868c9-8jw5w" Jul 02 13:53:01.180480 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.180303 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb695f8e-21bb-4604-a6fa-61e0a1b32150-openshift-service-ca\") pod \"maas-api-86f58868c9-8jw5w\" (UID: \"eb695f8e-21bb-4604-a6fa-61e0a1b32150\") " pod="opendatahub/maas-api-86f58868c9-8jw5w" Jul 02 13:53:01.180480 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.180326 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb695f8e-21bb-4604-a6fa-61e0a1b32150-maas-api-tls\") pod \"maas-api-86f58868c9-8jw5w\" (UID: \"eb695f8e-21bb-4604-a6fa-61e0a1b32150\") " pod="opendatahub/maas-api-86f58868c9-8jw5w" Jul 02 13:53:01.181079 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.181059 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eb695f8e-21bb-4604-a6fa-61e0a1b32150-openshift-service-ca\") pod \"maas-api-86f58868c9-8jw5w\" (UID: \"eb695f8e-21bb-4604-a6fa-61e0a1b32150\") " pod="opendatahub/maas-api-86f58868c9-8jw5w" Jul 02 13:53:01.182866 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.182849 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eb695f8e-21bb-4604-a6fa-61e0a1b32150-maas-api-tls\") pod \"maas-api-86f58868c9-8jw5w\" (UID: \"eb695f8e-21bb-4604-a6fa-61e0a1b32150\") " pod="opendatahub/maas-api-86f58868c9-8jw5w" Jul 02 13:53:01.281833 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.281734 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws5jr\" (UniqueName: \"kubernetes.io/projected/eb695f8e-21bb-4604-a6fa-61e0a1b32150-kube-api-access-ws5jr\") pod \"maas-api-86f58868c9-8jw5w\" (UID: \"eb695f8e-21bb-4604-a6fa-61e0a1b32150\") " pod="opendatahub/maas-api-86f58868c9-8jw5w" Jul 02 13:53:01.289530 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.289503 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws5jr\" (UniqueName: \"kubernetes.io/projected/eb695f8e-21bb-4604-a6fa-61e0a1b32150-kube-api-access-ws5jr\") pod \"maas-api-86f58868c9-8jw5w\" (UID: \"eb695f8e-21bb-4604-a6fa-61e0a1b32150\") " pod="opendatahub/maas-api-86f58868c9-8jw5w" Jul 02 13:53:01.313554 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.313517 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-86f58868c9-8jw5w" Jul 02 13:53:01.457611 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.457329 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-86f58868c9-8jw5w"] Jul 02 13:53:01.462038 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:53:01.462001 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb695f8e_21bb_4604_a6fa_61e0a1b32150.slice/crio-d6a70497af7320804ff4bd4accb0221bab5c16d83a726e13d73f88d2de5e2558 WatchSource:0}: Error finding container d6a70497af7320804ff4bd4accb0221bab5c16d83a726e13d73f88d2de5e2558: Status 404 returned error can't find the container with id d6a70497af7320804ff4bd4accb0221bab5c16d83a726e13d73f88d2de5e2558 Jul 02 13:53:01.768600 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.768566 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59f49fd4f6-8x5r9"] Jul 02 13:53:01.773543 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.773519 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-8x5r9" Jul 02 13:53:01.775761 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.775738 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 02 13:53:01.775907 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.775890 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-v6x9w\"" Jul 02 13:53:01.782349 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.782282 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59f49fd4f6-8x5r9"] Jul 02 13:53:01.784259 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.784224 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78rn6\" (UniqueName: \"kubernetes.io/projected/4e71ad19-1c20-4e48-8c7f-df4c410eb929-kube-api-access-78rn6\") pod \"payload-pre-processing-59f49fd4f6-8x5r9\" (UID: \"4e71ad19-1c20-4e48-8c7f-df4c410eb929\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-8x5r9" Jul 02 13:53:01.784405 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.784293 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e71ad19-1c20-4e48-8c7f-df4c410eb929-plugins-config-volume\") pod \"payload-pre-processing-59f49fd4f6-8x5r9\" (UID: \"4e71ad19-1c20-4e48-8c7f-df4c410eb929\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-8x5r9" Jul 02 13:53:01.827542 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.827505 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-86f58868c9-8jw5w" event={"ID":"eb695f8e-21bb-4604-a6fa-61e0a1b32150","Type":"ContainerStarted","Data":"d6a70497af7320804ff4bd4accb0221bab5c16d83a726e13d73f88d2de5e2558"} Jul 02 13:53:01.833802 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.833770 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-sqv8q"] Jul 02 13:53:01.839059 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.839023 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" Jul 02 13:53:01.848522 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.848493 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-sqv8q"] Jul 02 13:53:01.885221 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.885182 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78rn6\" (UniqueName: \"kubernetes.io/projected/4e71ad19-1c20-4e48-8c7f-df4c410eb929-kube-api-access-78rn6\") pod \"payload-pre-processing-59f49fd4f6-8x5r9\" (UID: \"4e71ad19-1c20-4e48-8c7f-df4c410eb929\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-8x5r9" Jul 02 13:53:01.885438 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.885290 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e71ad19-1c20-4e48-8c7f-df4c410eb929-plugins-config-volume\") pod \"payload-pre-processing-59f49fd4f6-8x5r9\" (UID: \"4e71ad19-1c20-4e48-8c7f-df4c410eb929\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-8x5r9" Jul 02 13:53:01.885438 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.885352 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6aefd846-e612-47b2-acf0-98746387507f-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-sqv8q\" (UID: \"6aefd846-e612-47b2-acf0-98746387507f\") " pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" Jul 02 13:53:01.885438 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.885411 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56zcf\" (UniqueName: \"kubernetes.io/projected/6aefd846-e612-47b2-acf0-98746387507f-kube-api-access-56zcf\") pod \"payload-processing-7bbcb654f4-sqv8q\" (UID: \"6aefd846-e612-47b2-acf0-98746387507f\") " pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" Jul 02 13:53:01.886292 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.886264 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e71ad19-1c20-4e48-8c7f-df4c410eb929-plugins-config-volume\") pod \"payload-pre-processing-59f49fd4f6-8x5r9\" (UID: \"4e71ad19-1c20-4e48-8c7f-df4c410eb929\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-8x5r9" Jul 02 13:53:01.905093 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.905068 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-78rn6\" (UniqueName: \"kubernetes.io/projected/4e71ad19-1c20-4e48-8c7f-df4c410eb929-kube-api-access-78rn6\") pod \"payload-pre-processing-59f49fd4f6-8x5r9\" (UID: \"4e71ad19-1c20-4e48-8c7f-df4c410eb929\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-8x5r9" Jul 02 13:53:01.986073 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.986037 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6aefd846-e612-47b2-acf0-98746387507f-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-sqv8q\" (UID: \"6aefd846-e612-47b2-acf0-98746387507f\") " pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" Jul 02 13:53:01.986247 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.986094 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56zcf\" (UniqueName: \"kubernetes.io/projected/6aefd846-e612-47b2-acf0-98746387507f-kube-api-access-56zcf\") pod \"payload-processing-7bbcb654f4-sqv8q\" (UID: \"6aefd846-e612-47b2-acf0-98746387507f\") " pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" Jul 02 13:53:01.986706 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.986688 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6aefd846-e612-47b2-acf0-98746387507f-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-sqv8q\" (UID: \"6aefd846-e612-47b2-acf0-98746387507f\") " pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" Jul 02 13:53:01.994288 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:01.994253 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56zcf\" (UniqueName: \"kubernetes.io/projected/6aefd846-e612-47b2-acf0-98746387507f-kube-api-access-56zcf\") pod \"payload-processing-7bbcb654f4-sqv8q\" (UID: \"6aefd846-e612-47b2-acf0-98746387507f\") " pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" Jul 02 13:53:02.087018 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:02.086905 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-8x5r9" Jul 02 13:53:02.151081 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:02.151052 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" Jul 02 13:53:02.232076 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:02.232003 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59f49fd4f6-8x5r9"] Jul 02 13:53:02.235203 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:53:02.235164 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e71ad19_1c20_4e48_8c7f_df4c410eb929.slice/crio-7eb5df4e0b377360a06e253c7127347a5b9559a2b00112f0f142fe74693b035a WatchSource:0}: Error finding container 7eb5df4e0b377360a06e253c7127347a5b9559a2b00112f0f142fe74693b035a: Status 404 returned error can't find the container with id 7eb5df4e0b377360a06e253c7127347a5b9559a2b00112f0f142fe74693b035a Jul 02 13:53:02.306266 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:02.306229 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-sqv8q"] Jul 02 13:53:02.310626 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:53:02.310588 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aefd846_e612_47b2_acf0_98746387507f.slice/crio-ebb55535d812f4669cf0c11e0b67efeb6948c102ec050822084324c13c621982 WatchSource:0}: Error finding container ebb55535d812f4669cf0c11e0b67efeb6948c102ec050822084324c13c621982: Status 404 returned error can't find the container with id ebb55535d812f4669cf0c11e0b67efeb6948c102ec050822084324c13c621982 Jul 02 13:53:02.834329 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:02.834287 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" event={"ID":"6aefd846-e612-47b2-acf0-98746387507f","Type":"ContainerStarted","Data":"ebb55535d812f4669cf0c11e0b67efeb6948c102ec050822084324c13c621982"} Jul 02 13:53:02.835644 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:02.835611 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-8x5r9" event={"ID":"4e71ad19-1c20-4e48-8c7f-df4c410eb929","Type":"ContainerStarted","Data":"7eb5df4e0b377360a06e253c7127347a5b9559a2b00112f0f142fe74693b035a"} Jul 02 13:53:03.845075 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:03.844963 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-86f58868c9-8jw5w" event={"ID":"eb695f8e-21bb-4604-a6fa-61e0a1b32150","Type":"ContainerStarted","Data":"ebd34121592e1307d787488c10796b2576c08bf711d80149c8612763d0b7d9a6"} Jul 02 13:53:03.845542 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:03.845210 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-86f58868c9-8jw5w" Jul 02 13:53:03.862139 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:03.862071 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-86f58868c9-8jw5w" podStartSLOduration=2.215554497 podStartE2EDuration="3.862054007s" totalImagesPullingTime="1.64649951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:53:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:53:03.861154336 +0000 UTC m=+689.987290845" watchObservedRunningTime="2026-07-02 13:53:03.862054007 +0000 UTC m=+689.988190512" Jul 02 13:53:05.858627 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:05.858590 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" event={"ID":"6aefd846-e612-47b2-acf0-98746387507f","Type":"ContainerStarted","Data":"05afcbd6647ef2e47bb4c25bfe44efcd967d5b5db70722dbd140a7fe9c6de4c4"} Jul 02 13:53:05.859115 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:05.858787 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" Jul 02 13:53:05.860220 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:05.860197 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-8x5r9" event={"ID":"4e71ad19-1c20-4e48-8c7f-df4c410eb929","Type":"ContainerStarted","Data":"3c1361dc21ee8ad2f99e9837a480ea3ef55dc970433dbf75c6aafc84e5280891"} Jul 02 13:53:05.860348 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:05.860263 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-8x5r9" Jul 02 13:53:05.878336 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:05.878291 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" podStartSLOduration=2.109077548 podStartE2EDuration="4.878277418s" totalImagesPullingTime="2.76919987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:53:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:53:05.876043271 +0000 UTC m=+692.002179778" watchObservedRunningTime="2026-07-02 13:53:05.878277418 +0000 UTC m=+692.004413922" Jul 02 13:53:05.894590 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:05.894534 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-8x5r9" podStartSLOduration=2.0439244 podStartE2EDuration="4.89451558s" totalImagesPullingTime="2.85059118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:53:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:53:05.893908704 +0000 UTC m=+692.020045234" watchObservedRunningTime="2026-07-02 13:53:05.89451558 +0000 UTC m=+692.020652093" Jul 02 13:53:09.856662 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:09.856636 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-86f58868c9-8jw5w" Jul 02 13:53:11.325455 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:11.325417 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 02 13:53:11.325855 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:11.325605 2561 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="76a4dbe8-f8a0-4e05-b358-82d17125c254" containerName="keycloak" containerID="cri-o://d48665fb46ebe70c337dbc1d47072dada6eeab4d28da3b26183115f20bbcbbed" gracePeriod=30 Jul 02 13:53:12.900644 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:12.900609 2561 generic.go:356] "Generic (PLEG): container finished" podID="76a4dbe8-f8a0-4e05-b358-82d17125c254" containerID="d48665fb46ebe70c337dbc1d47072dada6eeab4d28da3b26183115f20bbcbbed" exitCode=143 Jul 02 13:53:12.901043 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:12.900682 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"76a4dbe8-f8a0-4e05-b358-82d17125c254","Type":"ContainerDied","Data":"d48665fb46ebe70c337dbc1d47072dada6eeab4d28da3b26183115f20bbcbbed"} Jul 02 13:53:13.383554 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.383527 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 02 13:53:13.488520 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.488481 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kxv6\" (UniqueName: \"kubernetes.io/projected/76a4dbe8-f8a0-4e05-b358-82d17125c254-kube-api-access-2kxv6\") pod \"76a4dbe8-f8a0-4e05-b358-82d17125c254\" (UID: \"76a4dbe8-f8a0-4e05-b358-82d17125c254\") " Jul 02 13:53:13.490765 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.490687 2561 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76a4dbe8-f8a0-4e05-b358-82d17125c254-kube-api-access-2kxv6" (OuterVolumeSpecName: "kube-api-access-2kxv6") pod "76a4dbe8-f8a0-4e05-b358-82d17125c254" (UID: "76a4dbe8-f8a0-4e05-b358-82d17125c254"). InnerVolumeSpecName "kube-api-access-2kxv6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 13:53:13.590033 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.589998 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kxv6\" (UniqueName: \"kubernetes.io/projected/76a4dbe8-f8a0-4e05-b358-82d17125c254-kube-api-access-2kxv6\") on node \"ip-10-0-141-112.ec2.internal\" DevicePath \"\"" Jul 02 13:53:13.907803 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.907728 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 02 13:53:13.908301 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.907728 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"76a4dbe8-f8a0-4e05-b358-82d17125c254","Type":"ContainerDied","Data":"70cf81013b9267f288f350a8c5df38e3c9c80d30369ca2507db0976dd4e84380"} Jul 02 13:53:13.908301 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.907842 2561 scope.go:117] "RemoveContainer" containerID="d48665fb46ebe70c337dbc1d47072dada6eeab4d28da3b26183115f20bbcbbed" Jul 02 13:53:13.929060 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.929026 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 02 13:53:13.933123 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.933094 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 02 13:53:13.956008 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.955954 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 02 13:53:13.956525 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.956490 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76a4dbe8-f8a0-4e05-b358-82d17125c254" containerName="keycloak" Jul 02 13:53:13.956525 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.956512 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="76a4dbe8-f8a0-4e05-b358-82d17125c254" containerName="keycloak" Jul 02 13:53:13.956695 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.956655 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="76a4dbe8-f8a0-4e05-b358-82d17125c254" containerName="keycloak" Jul 02 13:53:13.961092 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.961069 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 02 13:53:13.964041 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.964013 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jul 02 13:53:13.964179 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.964047 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jul 02 13:53:13.964179 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.964050 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jul 02 13:53:13.964179 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.964117 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jul 02 13:53:13.964344 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.964290 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-tj4wh\"" Jul 02 13:53:13.972299 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.972271 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 02 13:53:13.992741 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.992709 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnb9q\" (UniqueName: \"kubernetes.io/projected/59a2a9ba-8e05-462e-8931-68f11054fa22-kube-api-access-rnb9q\") pod \"maas-keycloak-0\" (UID: \"59a2a9ba-8e05-462e-8931-68f11054fa22\") " pod="keycloak-system/maas-keycloak-0" Jul 02 13:53:13.992917 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:13.992775 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/59a2a9ba-8e05-462e-8931-68f11054fa22-test-realms\") pod \"maas-keycloak-0\" (UID: \"59a2a9ba-8e05-462e-8931-68f11054fa22\") " pod="keycloak-system/maas-keycloak-0" Jul 02 13:53:14.094314 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:14.094272 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnb9q\" (UniqueName: \"kubernetes.io/projected/59a2a9ba-8e05-462e-8931-68f11054fa22-kube-api-access-rnb9q\") pod \"maas-keycloak-0\" (UID: \"59a2a9ba-8e05-462e-8931-68f11054fa22\") " pod="keycloak-system/maas-keycloak-0" Jul 02 13:53:14.094491 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:14.094350 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/59a2a9ba-8e05-462e-8931-68f11054fa22-test-realms\") pod \"maas-keycloak-0\" (UID: \"59a2a9ba-8e05-462e-8931-68f11054fa22\") " pod="keycloak-system/maas-keycloak-0" Jul 02 13:53:14.095048 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:14.095031 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/59a2a9ba-8e05-462e-8931-68f11054fa22-test-realms\") pod \"maas-keycloak-0\" (UID: \"59a2a9ba-8e05-462e-8931-68f11054fa22\") " pod="keycloak-system/maas-keycloak-0" Jul 02 13:53:14.102508 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:14.102487 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnb9q\" (UniqueName: \"kubernetes.io/projected/59a2a9ba-8e05-462e-8931-68f11054fa22-kube-api-access-rnb9q\") pod \"maas-keycloak-0\" (UID: \"59a2a9ba-8e05-462e-8931-68f11054fa22\") " pod="keycloak-system/maas-keycloak-0" Jul 02 13:53:14.272258 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:14.272227 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 02 13:53:14.366270 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:14.366238 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76a4dbe8-f8a0-4e05-b358-82d17125c254" path="/var/lib/kubelet/pods/76a4dbe8-f8a0-4e05-b358-82d17125c254/volumes" Jul 02 13:53:14.421459 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:14.421294 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 02 13:53:14.424465 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:53:14.424431 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59a2a9ba_8e05_462e_8931_68f11054fa22.slice/crio-c4b3c9b7ce192e00c75be50a4b73f40138314c3db52eb8bd013c539576c4b1da WatchSource:0}: Error finding container c4b3c9b7ce192e00c75be50a4b73f40138314c3db52eb8bd013c539576c4b1da: Status 404 returned error can't find the container with id c4b3c9b7ce192e00c75be50a4b73f40138314c3db52eb8bd013c539576c4b1da Jul 02 13:53:14.916472 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:14.916437 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"59a2a9ba-8e05-462e-8931-68f11054fa22","Type":"ContainerStarted","Data":"2de584310f5c61e08cb799a2606902ab6556f1c861f2ba224a69d74628f2d4a4"} Jul 02 13:53:14.916863 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:14.916496 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"59a2a9ba-8e05-462e-8931-68f11054fa22","Type":"ContainerStarted","Data":"c4b3c9b7ce192e00c75be50a4b73f40138314c3db52eb8bd013c539576c4b1da"} Jul 02 13:53:14.935550 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:14.935386 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.5285628839999998 podStartE2EDuration="1.935366519s" totalImagesPullingTime="406.803635ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:53:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:53:14.934773801 +0000 UTC m=+701.060910320" watchObservedRunningTime="2026-07-02 13:53:14.935366519 +0000 UTC m=+701.061503027" Jul 02 13:53:15.272894 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:15.272852 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jul 02 13:53:15.274527 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:15.274490 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59a2a9ba-8e05-462e-8931-68f11054fa22" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jul 02 13:53:16.273576 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:16.273516 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59a2a9ba-8e05-462e-8931-68f11054fa22" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jul 02 13:53:16.868026 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:16.867958 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-8x5r9" Jul 02 13:53:16.868238 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:16.868131 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" Jul 02 13:53:17.273527 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:17.273476 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59a2a9ba-8e05-462e-8931-68f11054fa22" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jul 02 13:53:18.273273 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:18.273218 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59a2a9ba-8e05-462e-8931-68f11054fa22" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jul 02 13:53:19.273731 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:19.273682 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59a2a9ba-8e05-462e-8931-68f11054fa22" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jul 02 13:53:20.273542 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:20.273498 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59a2a9ba-8e05-462e-8931-68f11054fa22" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jul 02 13:53:21.273816 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:21.273768 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59a2a9ba-8e05-462e-8931-68f11054fa22" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jul 02 13:53:22.273990 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:22.273930 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59a2a9ba-8e05-462e-8931-68f11054fa22" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jul 02 13:53:23.273688 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:23.273636 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59a2a9ba-8e05-462e-8931-68f11054fa22" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jul 02 13:53:24.272613 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:24.272567 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jul 02 13:53:24.273117 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:24.272965 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59a2a9ba-8e05-462e-8931-68f11054fa22" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jul 02 13:53:25.272712 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:25.272661 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59a2a9ba-8e05-462e-8931-68f11054fa22" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jul 02 13:53:26.273785 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:26.273733 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59a2a9ba-8e05-462e-8931-68f11054fa22" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jul 02 13:53:27.273170 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:27.273130 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59a2a9ba-8e05-462e-8931-68f11054fa22" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jul 02 13:53:28.273009 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:28.272941 2561 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="59a2a9ba-8e05-462e-8931-68f11054fa22" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.42:9000/health/started\": dial tcp 10.134.0.42:9000: connect: connection refused" Jul 02 13:53:29.404144 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:29.404104 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jul 02 13:53:29.423907 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:29.423860 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="59a2a9ba-8e05-462e-8931-68f11054fa22" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 13:53:39.410184 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:53:39.410148 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jul 02 13:54:02.902164 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:02.902126 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45"] Jul 02 13:54:02.905645 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:02.905623 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:02.909883 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:02.909857 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-z7nwb\"" Jul 02 13:54:02.910043 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:02.909915 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 02 13:54:02.910496 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:02.910482 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 02 13:54:02.919193 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:02.919169 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 02 13:54:02.924600 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:02.924562 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45"] Jul 02 13:54:02.981085 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:02.981052 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acec7ef3-d3a7-45e5-9968-314ff0cfdf19-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45\" (UID: \"acec7ef3-d3a7-45e5-9968-314ff0cfdf19\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:02.981085 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:02.981084 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acec7ef3-d3a7-45e5-9968-314ff0cfdf19-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45\" (UID: \"acec7ef3-d3a7-45e5-9968-314ff0cfdf19\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:02.981322 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:02.981120 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acec7ef3-d3a7-45e5-9968-314ff0cfdf19-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45\" (UID: \"acec7ef3-d3a7-45e5-9968-314ff0cfdf19\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:02.981322 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:02.981220 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acec7ef3-d3a7-45e5-9968-314ff0cfdf19-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45\" (UID: \"acec7ef3-d3a7-45e5-9968-314ff0cfdf19\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:02.981322 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:02.981292 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acec7ef3-d3a7-45e5-9968-314ff0cfdf19-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45\" (UID: \"acec7ef3-d3a7-45e5-9968-314ff0cfdf19\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:02.981322 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:02.981313 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m44dx\" (UniqueName: \"kubernetes.io/projected/acec7ef3-d3a7-45e5-9968-314ff0cfdf19-kube-api-access-m44dx\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45\" (UID: \"acec7ef3-d3a7-45e5-9968-314ff0cfdf19\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:03.082042 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:03.082010 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acec7ef3-d3a7-45e5-9968-314ff0cfdf19-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45\" (UID: \"acec7ef3-d3a7-45e5-9968-314ff0cfdf19\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:03.082212 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:03.082046 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m44dx\" (UniqueName: \"kubernetes.io/projected/acec7ef3-d3a7-45e5-9968-314ff0cfdf19-kube-api-access-m44dx\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45\" (UID: \"acec7ef3-d3a7-45e5-9968-314ff0cfdf19\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:03.082212 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:03.082087 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acec7ef3-d3a7-45e5-9968-314ff0cfdf19-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45\" (UID: \"acec7ef3-d3a7-45e5-9968-314ff0cfdf19\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:03.082212 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:03.082104 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acec7ef3-d3a7-45e5-9968-314ff0cfdf19-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45\" (UID: \"acec7ef3-d3a7-45e5-9968-314ff0cfdf19\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:03.082212 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:03.082154 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acec7ef3-d3a7-45e5-9968-314ff0cfdf19-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45\" (UID: \"acec7ef3-d3a7-45e5-9968-314ff0cfdf19\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:03.082365 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:03.082213 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acec7ef3-d3a7-45e5-9968-314ff0cfdf19-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45\" (UID: \"acec7ef3-d3a7-45e5-9968-314ff0cfdf19\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:03.082528 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:03.082508 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/acec7ef3-d3a7-45e5-9968-314ff0cfdf19-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45\" (UID: \"acec7ef3-d3a7-45e5-9968-314ff0cfdf19\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:03.082593 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:03.082563 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/acec7ef3-d3a7-45e5-9968-314ff0cfdf19-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45\" (UID: \"acec7ef3-d3a7-45e5-9968-314ff0cfdf19\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:03.082593 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:03.082558 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/acec7ef3-d3a7-45e5-9968-314ff0cfdf19-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45\" (UID: \"acec7ef3-d3a7-45e5-9968-314ff0cfdf19\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:03.084524 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:03.084497 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/acec7ef3-d3a7-45e5-9968-314ff0cfdf19-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45\" (UID: \"acec7ef3-d3a7-45e5-9968-314ff0cfdf19\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:03.084702 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:03.084686 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/acec7ef3-d3a7-45e5-9968-314ff0cfdf19-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45\" (UID: \"acec7ef3-d3a7-45e5-9968-314ff0cfdf19\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:03.103019 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:03.102990 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m44dx\" (UniqueName: \"kubernetes.io/projected/acec7ef3-d3a7-45e5-9968-314ff0cfdf19-kube-api-access-m44dx\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45\" (UID: \"acec7ef3-d3a7-45e5-9968-314ff0cfdf19\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:03.217138 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:03.217086 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:03.354223 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:03.354196 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45"] Jul 02 13:54:03.356362 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:54:03.356333 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacec7ef3_d3a7_45e5_9968_314ff0cfdf19.slice/crio-25b0e8db529f2db10f79789f8e905cacce7eda6e594161bd74bb260129f79931 WatchSource:0}: Error finding container 25b0e8db529f2db10f79789f8e905cacce7eda6e594161bd74bb260129f79931: Status 404 returned error can't find the container with id 25b0e8db529f2db10f79789f8e905cacce7eda6e594161bd74bb260129f79931 Jul 02 13:54:04.236710 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:04.236662 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" event={"ID":"acec7ef3-d3a7-45e5-9968-314ff0cfdf19","Type":"ContainerStarted","Data":"25b0e8db529f2db10f79789f8e905cacce7eda6e594161bd74bb260129f79931"} Jul 02 13:54:05.243823 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:05.243785 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" event={"ID":"acec7ef3-d3a7-45e5-9968-314ff0cfdf19","Type":"ContainerStarted","Data":"898a118a44e43c726065e00b04af1d3d465a7f40e68a8c581c434dfe2fadef03"} Jul 02 13:54:05.263070 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:05.262997 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" podStartSLOduration=1.809034083 podStartE2EDuration="3.262949171s" totalImagesPullingTime="1.453915088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:54:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:54:05.262629274 +0000 UTC m=+751.388765793" watchObservedRunningTime="2026-07-02 13:54:05.262949171 +0000 UTC m=+751.389085678" Jul 02 13:54:13.218072 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:13.218023 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:13.218503 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:13.218107 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:13.230644 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:13.230618 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:13.304347 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:13.304301 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45" Jul 02 13:54:24.691427 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.691391 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x"] Jul 02 13:54:24.698078 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.698057 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:24.700465 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.700444 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 02 13:54:24.705948 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.705924 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x"] Jul 02 13:54:24.802287 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.802254 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae335c1e-7897-4b1c-bdde-7f4e176efbc9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x\" (UID: \"ae335c1e-7897-4b1c-bdde-7f4e176efbc9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:24.802449 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.802309 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae335c1e-7897-4b1c-bdde-7f4e176efbc9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x\" (UID: \"ae335c1e-7897-4b1c-bdde-7f4e176efbc9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:24.802449 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.802369 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49hd2\" (UniqueName: \"kubernetes.io/projected/ae335c1e-7897-4b1c-bdde-7f4e176efbc9-kube-api-access-49hd2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x\" (UID: \"ae335c1e-7897-4b1c-bdde-7f4e176efbc9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:24.802449 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.802396 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae335c1e-7897-4b1c-bdde-7f4e176efbc9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x\" (UID: \"ae335c1e-7897-4b1c-bdde-7f4e176efbc9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:24.802449 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.802415 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae335c1e-7897-4b1c-bdde-7f4e176efbc9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x\" (UID: \"ae335c1e-7897-4b1c-bdde-7f4e176efbc9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:24.802449 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.802448 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae335c1e-7897-4b1c-bdde-7f4e176efbc9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x\" (UID: \"ae335c1e-7897-4b1c-bdde-7f4e176efbc9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:24.903656 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.903612 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae335c1e-7897-4b1c-bdde-7f4e176efbc9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x\" (UID: \"ae335c1e-7897-4b1c-bdde-7f4e176efbc9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:24.903851 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.903672 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae335c1e-7897-4b1c-bdde-7f4e176efbc9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x\" (UID: \"ae335c1e-7897-4b1c-bdde-7f4e176efbc9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:24.903851 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.903709 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49hd2\" (UniqueName: \"kubernetes.io/projected/ae335c1e-7897-4b1c-bdde-7f4e176efbc9-kube-api-access-49hd2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x\" (UID: \"ae335c1e-7897-4b1c-bdde-7f4e176efbc9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:24.903851 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.903732 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae335c1e-7897-4b1c-bdde-7f4e176efbc9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x\" (UID: \"ae335c1e-7897-4b1c-bdde-7f4e176efbc9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:24.903851 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.903753 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae335c1e-7897-4b1c-bdde-7f4e176efbc9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x\" (UID: \"ae335c1e-7897-4b1c-bdde-7f4e176efbc9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:24.903851 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.903802 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae335c1e-7897-4b1c-bdde-7f4e176efbc9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x\" (UID: \"ae335c1e-7897-4b1c-bdde-7f4e176efbc9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:24.904187 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.904162 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae335c1e-7897-4b1c-bdde-7f4e176efbc9-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x\" (UID: \"ae335c1e-7897-4b1c-bdde-7f4e176efbc9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:24.904265 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.904184 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae335c1e-7897-4b1c-bdde-7f4e176efbc9-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x\" (UID: \"ae335c1e-7897-4b1c-bdde-7f4e176efbc9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:24.904324 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.904286 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae335c1e-7897-4b1c-bdde-7f4e176efbc9-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x\" (UID: \"ae335c1e-7897-4b1c-bdde-7f4e176efbc9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:24.905954 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.905936 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae335c1e-7897-4b1c-bdde-7f4e176efbc9-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x\" (UID: \"ae335c1e-7897-4b1c-bdde-7f4e176efbc9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:24.906151 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.906134 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae335c1e-7897-4b1c-bdde-7f4e176efbc9-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x\" (UID: \"ae335c1e-7897-4b1c-bdde-7f4e176efbc9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:24.916924 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:24.916895 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49hd2\" (UniqueName: \"kubernetes.io/projected/ae335c1e-7897-4b1c-bdde-7f4e176efbc9-kube-api-access-49hd2\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x\" (UID: \"ae335c1e-7897-4b1c-bdde-7f4e176efbc9\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:25.009731 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:25.009688 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:25.155564 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:25.155535 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x"] Jul 02 13:54:25.157860 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:54:25.157836 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae335c1e_7897_4b1c_bdde_7f4e176efbc9.slice/crio-53ccc1e74c7a12aa17f299a10c9888d9763b5e214618c8b5f8a1c2b57b3b68b7 WatchSource:0}: Error finding container 53ccc1e74c7a12aa17f299a10c9888d9763b5e214618c8b5f8a1c2b57b3b68b7: Status 404 returned error can't find the container with id 53ccc1e74c7a12aa17f299a10c9888d9763b5e214618c8b5f8a1c2b57b3b68b7 Jul 02 13:54:25.352034 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:25.351910 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" event={"ID":"ae335c1e-7897-4b1c-bdde-7f4e176efbc9","Type":"ContainerStarted","Data":"53ccc1e74c7a12aa17f299a10c9888d9763b5e214618c8b5f8a1c2b57b3b68b7"} Jul 02 13:54:26.363634 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:26.363599 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" event={"ID":"ae335c1e-7897-4b1c-bdde-7f4e176efbc9","Type":"ContainerStarted","Data":"ddf20a4aa18d4a2d97e33b584e0ab5761b4c2ccc967c262ba58f202c61b44d1b"} Jul 02 13:54:26.386752 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:26.386704 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" podStartSLOduration=2.135284216 podStartE2EDuration="2.386686875s" totalImagesPullingTime="251.402659ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 13:54:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:54:26.382013091 +0000 UTC m=+772.508149590" watchObservedRunningTime="2026-07-02 13:54:26.386686875 +0000 UTC m=+772.512823385" Jul 02 13:54:35.010051 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:35.010013 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:35.010482 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:35.010077 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:35.022853 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:35.022821 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:54:35.415007 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:54:35.414915 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x" Jul 02 13:55:32.706027 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:55:32.705959 2561 generic.go:356] "Generic (PLEG): container finished" podID="6aefd846-e612-47b2-acf0-98746387507f" containerID="05afcbd6647ef2e47bb4c25bfe44efcd967d5b5db70722dbd140a7fe9c6de4c4" exitCode=2 Jul 02 13:55:32.706598 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:55:32.706027 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" event={"ID":"6aefd846-e612-47b2-acf0-98746387507f","Type":"ContainerDied","Data":"05afcbd6647ef2e47bb4c25bfe44efcd967d5b5db70722dbd140a7fe9c6de4c4"} Jul 02 13:55:32.706598 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:55:32.706391 2561 scope.go:117] "RemoveContainer" containerID="05afcbd6647ef2e47bb4c25bfe44efcd967d5b5db70722dbd140a7fe9c6de4c4" Jul 02 13:55:33.713365 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:55:33.713329 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" event={"ID":"6aefd846-e612-47b2-acf0-98746387507f","Type":"ContainerStarted","Data":"6f4831e8a62fc09ab0db3c404ed20efc2d01a75f1b1de7a24ac84f4691ef330b"} Jul 02 13:55:33.713832 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:55:33.713534 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" Jul 02 13:55:44.721351 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:55:44.721314 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" Jul 02 13:56:05.876592 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:05.876489 2561 generic.go:356] "Generic (PLEG): container finished" podID="6aefd846-e612-47b2-acf0-98746387507f" containerID="6f4831e8a62fc09ab0db3c404ed20efc2d01a75f1b1de7a24ac84f4691ef330b" exitCode=2 Jul 02 13:56:05.876592 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:05.876573 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" event={"ID":"6aefd846-e612-47b2-acf0-98746387507f","Type":"ContainerDied","Data":"6f4831e8a62fc09ab0db3c404ed20efc2d01a75f1b1de7a24ac84f4691ef330b"} Jul 02 13:56:05.877183 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:05.876660 2561 scope.go:117] "RemoveContainer" containerID="05afcbd6647ef2e47bb4c25bfe44efcd967d5b5db70722dbd140a7fe9c6de4c4" Jul 02 13:56:05.877183 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:05.877050 2561 scope.go:117] "RemoveContainer" containerID="6f4831e8a62fc09ab0db3c404ed20efc2d01a75f1b1de7a24ac84f4691ef330b" Jul 02 13:56:05.877331 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:56:05.877308 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 10s restarting failed container=payload-processing pod=payload-processing-7bbcb654f4-sqv8q_openshift-ingress(6aefd846-e612-47b2-acf0-98746387507f)\"" pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" podUID="6aefd846-e612-47b2-acf0-98746387507f" Jul 02 13:56:10.885689 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:10.885659 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-86f58868c9-8jw5w_eb695f8e-21bb-4604-a6fa-61e0a1b32150/maas-api/0.log" Jul 02 13:56:11.004753 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:11.004706 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-56c678849-slc8b_ed8b896e-29aa-4383-ae97-57911e4c99ff/manager/0.log" Jul 02 13:56:11.498840 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:11.498809 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5964c6974c-sfjzz_9f18bedb-22f1-43db-be9e-a7012ff27142/copy-manifests/0.log" Jul 02 13:56:11.505593 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:11.505566 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5964c6974c-sfjzz_9f18bedb-22f1-43db-be9e-a7012ff27142/manager/0.log" Jul 02 13:56:11.741112 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:11.741082 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-88cc6_f496bfe6-dbd4-405c-90de-7239d985a474/postgres/0.log" Jul 02 13:56:13.422476 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:13.422448 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-x98tr_e50a83c3-13f2-407b-bb01-0944ac952a86/kuadrant-console-plugin/0.log" Jul 02 13:56:14.658291 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:14.658263 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-685667f9f9-4rcj4_c5698427-1618-4369-9d7c-82c74c7a27d1/kube-auth-proxy/0.log" Jul 02 13:56:14.719546 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:14.719503 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" Jul 02 13:56:14.719907 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:14.719893 2561 scope.go:117] "RemoveContainer" containerID="6f4831e8a62fc09ab0db3c404ed20efc2d01a75f1b1de7a24ac84f4691ef330b" Jul 02 13:56:14.720146 ip-10-0-141-112 kubenswrapper[2561]: E0702 13:56:14.720126 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 10s restarting failed container=payload-processing pod=payload-processing-7bbcb654f4-sqv8q_openshift-ingress(6aefd846-e612-47b2-acf0-98746387507f)\"" pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" podUID="6aefd846-e612-47b2-acf0-98746387507f" Jul 02 13:56:14.900017 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:14.899992 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-8x5r9_4e71ad19-1c20-4e48-8c7f-df4c410eb929/payload-pre-processing/0.log" Jul 02 13:56:15.032932 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:15.032899 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-sqv8q_6aefd846-e612-47b2-acf0-98746387507f/payload-processing/1.log" Jul 02 13:56:15.515937 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:15.515889 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-mdb45_acec7ef3-d3a7-45e5-9968-314ff0cfdf19/main/0.log" Jul 02 13:56:15.869784 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:15.869706 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbchk8x_ae335c1e-7897-4b1c-bdde-7f4e176efbc9/main/0.log" Jul 02 13:56:22.151632 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:22.151594 2561 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" Jul 02 13:56:22.152090 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:22.151963 2561 scope.go:117] "RemoveContainer" containerID="6f4831e8a62fc09ab0db3c404ed20efc2d01a75f1b1de7a24ac84f4691ef330b" Jul 02 13:56:22.970784 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:22.970752 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" event={"ID":"6aefd846-e612-47b2-acf0-98746387507f","Type":"ContainerStarted","Data":"e3ba849218c2dd2bda738e75b75aa1e01237599ceab008a3cd991e955b736203"} Jul 02 13:56:22.971012 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:22.970994 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" Jul 02 13:56:24.060787 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:24.060755 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-flv4b_cf9efe36-5e20-4b2a-bda2-5eb6117bcb3f/global-pull-secret-syncer/0.log" Jul 02 13:56:24.177454 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:24.177424 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gp6w6_bf8c5897-0051-41d8-bc25-2f990a7dd74b/konnectivity-agent/0.log" Jul 02 13:56:24.251128 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:24.251091 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-112.ec2.internal_8f7c1481fcbae9bd1ac61a7f8b2618a4/haproxy/0.log" Jul 02 13:56:28.933172 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:28.933134 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-x98tr_e50a83c3-13f2-407b-bb01-0944ac952a86/kuadrant-console-plugin/0.log" Jul 02 13:56:30.804350 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:30.804320 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_58dfdeb1-5afc-4881-8dd2-3ee2964032c2/alertmanager/0.log" Jul 02 13:56:30.827544 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:30.827511 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_58dfdeb1-5afc-4881-8dd2-3ee2964032c2/config-reloader/0.log" Jul 02 13:56:30.855342 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:30.855316 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_58dfdeb1-5afc-4881-8dd2-3ee2964032c2/kube-rbac-proxy-web/0.log" Jul 02 13:56:30.877668 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:30.877644 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_58dfdeb1-5afc-4881-8dd2-3ee2964032c2/kube-rbac-proxy/0.log" Jul 02 13:56:30.903223 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:30.903202 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_58dfdeb1-5afc-4881-8dd2-3ee2964032c2/kube-rbac-proxy-metric/0.log" Jul 02 13:56:30.925904 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:30.925877 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_58dfdeb1-5afc-4881-8dd2-3ee2964032c2/prom-label-proxy/0.log" Jul 02 13:56:30.951752 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:30.951726 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_58dfdeb1-5afc-4881-8dd2-3ee2964032c2/init-config-reloader/0.log" Jul 02 13:56:31.011162 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.011136 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-8mr8c_62c2d08c-f0b1-4860-910f-48aab8a5101f/kube-state-metrics/0.log" Jul 02 13:56:31.031076 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.031047 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-8mr8c_62c2d08c-f0b1-4860-910f-48aab8a5101f/kube-rbac-proxy-main/0.log" Jul 02 13:56:31.056584 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.056514 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-8mr8c_62c2d08c-f0b1-4860-910f-48aab8a5101f/kube-rbac-proxy-self/0.log" Jul 02 13:56:31.103169 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.103141 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-864ff6486b-zhmhz_716d8090-1589-4a66-9784-fdd03c5e4358/metrics-server/0.log" Jul 02 13:56:31.133410 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.133385 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-cd8656b8-sg225_8e465295-97e6-498f-a211-e54e6bad7273/monitoring-plugin/0.log" Jul 02 13:56:31.447290 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.447266 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tp7m9_15039e2a-095d-4e7a-96aa-45ab21a5b491/node-exporter/0.log" Jul 02 13:56:31.471471 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.471442 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tp7m9_15039e2a-095d-4e7a-96aa-45ab21a5b491/kube-rbac-proxy/0.log" Jul 02 13:56:31.498674 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.498645 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tp7m9_15039e2a-095d-4e7a-96aa-45ab21a5b491/init-textfile/0.log" Jul 02 13:56:31.531725 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.531701 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-h6zdq_4d282969-fbb0-425e-a612-9a5801ed1b30/kube-rbac-proxy-main/0.log" Jul 02 13:56:31.568756 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.568731 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-h6zdq_4d282969-fbb0-425e-a612-9a5801ed1b30/kube-rbac-proxy-self/0.log" Jul 02 13:56:31.595480 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.595452 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-57f99c8b49-h6zdq_4d282969-fbb0-425e-a612-9a5801ed1b30/openshift-state-metrics/0.log" Jul 02 13:56:31.639425 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.639352 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8b11f3c5-7837-46ca-8ddf-bb25be65d879/prometheus/0.log" Jul 02 13:56:31.665696 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.665650 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8b11f3c5-7837-46ca-8ddf-bb25be65d879/config-reloader/0.log" Jul 02 13:56:31.690790 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.690763 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8b11f3c5-7837-46ca-8ddf-bb25be65d879/thanos-sidecar/0.log" Jul 02 13:56:31.715117 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.715087 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8b11f3c5-7837-46ca-8ddf-bb25be65d879/kube-rbac-proxy-web/0.log" Jul 02 13:56:31.740097 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.740078 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8b11f3c5-7837-46ca-8ddf-bb25be65d879/kube-rbac-proxy/0.log" Jul 02 13:56:31.761820 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.761764 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8b11f3c5-7837-46ca-8ddf-bb25be65d879/kube-rbac-proxy-thanos/0.log" Jul 02 13:56:31.788861 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.788831 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8b11f3c5-7837-46ca-8ddf-bb25be65d879/init-config-reloader/0.log" Jul 02 13:56:31.828774 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.828748 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-sz9zz_85818307-0372-452d-b408-3b21c8fafb90/prometheus-operator/0.log" Jul 02 13:56:31.855925 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.855904 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-sz9zz_85818307-0372-452d-b408-3b21c8fafb90/kube-rbac-proxy/0.log" Jul 02 13:56:31.882710 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.882664 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7ddc8784d4-kd72q_c23c15a1-a7a9-4493-b417-f7b099854dc7/prometheus-operator-admission-webhook/0.log" Jul 02 13:56:31.919890 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.919813 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-546998c87b-ktm8r_17574a8b-b188-4b49-8fb9-8ee93bc54bbf/telemeter-client/0.log" Jul 02 13:56:31.950216 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.950194 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-546998c87b-ktm8r_17574a8b-b188-4b49-8fb9-8ee93bc54bbf/reload/0.log" Jul 02 13:56:31.977182 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:31.977155 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-546998c87b-ktm8r_17574a8b-b188-4b49-8fb9-8ee93bc54bbf/kube-rbac-proxy/0.log" Jul 02 13:56:32.014726 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:32.014701 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d4bbc8bd-bb7t7_a783602a-9039-4d41-ad1f-13cedff176cf/thanos-query/0.log" Jul 02 13:56:32.042304 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:32.042276 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d4bbc8bd-bb7t7_a783602a-9039-4d41-ad1f-13cedff176cf/kube-rbac-proxy-web/0.log" Jul 02 13:56:32.065023 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:32.064993 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d4bbc8bd-bb7t7_a783602a-9039-4d41-ad1f-13cedff176cf/kube-rbac-proxy/0.log" Jul 02 13:56:32.086817 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:32.086789 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d4bbc8bd-bb7t7_a783602a-9039-4d41-ad1f-13cedff176cf/prom-label-proxy/0.log" Jul 02 13:56:32.110805 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:32.110775 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d4bbc8bd-bb7t7_a783602a-9039-4d41-ad1f-13cedff176cf/kube-rbac-proxy-rules/0.log" Jul 02 13:56:32.135316 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:32.135281 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d4bbc8bd-bb7t7_a783602a-9039-4d41-ad1f-13cedff176cf/kube-rbac-proxy-metrics/0.log" Jul 02 13:56:33.978569 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:33.978536 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bbcb654f4-sqv8q" Jul 02 13:56:34.588107 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.588066 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-755c49bf7-js9r8_8cc75d63-ff98-4268-ab92-b74dddfb4bd8/console/0.log" Jul 02 13:56:34.620160 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.620129 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5499b6fb8f-j4blc_c0d17ca3-aa6c-4dce-a610-b7c009989a2f/download-server/0.log" Jul 02 13:56:34.693548 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.693508 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9"] Jul 02 13:56:34.697732 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.697707 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" Jul 02 13:56:34.700114 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.700088 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pk25w\"/\"openshift-service-ca.crt\"" Jul 02 13:56:34.700805 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.700784 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pk25w\"/\"default-dockercfg-q9h4c\"" Jul 02 13:56:34.700910 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.700814 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pk25w\"/\"kube-root-ca.crt\"" Jul 02 13:56:34.705856 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.705832 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9"] Jul 02 13:56:34.788250 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.788209 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1e71a1c4-63a1-482e-bb06-510b2be28dab-podres\") pod \"perf-node-gather-daemonset-nbzv9\" (UID: \"1e71a1c4-63a1-482e-bb06-510b2be28dab\") " pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" Jul 02 13:56:34.788435 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.788361 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnhdm\" (UniqueName: \"kubernetes.io/projected/1e71a1c4-63a1-482e-bb06-510b2be28dab-kube-api-access-tnhdm\") pod \"perf-node-gather-daemonset-nbzv9\" (UID: \"1e71a1c4-63a1-482e-bb06-510b2be28dab\") " pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" Jul 02 13:56:34.788435 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.788419 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e71a1c4-63a1-482e-bb06-510b2be28dab-sys\") pod \"perf-node-gather-daemonset-nbzv9\" (UID: \"1e71a1c4-63a1-482e-bb06-510b2be28dab\") " pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" Jul 02 13:56:34.788543 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.788472 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e71a1c4-63a1-482e-bb06-510b2be28dab-lib-modules\") pod \"perf-node-gather-daemonset-nbzv9\" (UID: \"1e71a1c4-63a1-482e-bb06-510b2be28dab\") " pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" Jul 02 13:56:34.788543 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.788528 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1e71a1c4-63a1-482e-bb06-510b2be28dab-proc\") pod \"perf-node-gather-daemonset-nbzv9\" (UID: \"1e71a1c4-63a1-482e-bb06-510b2be28dab\") " pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" Jul 02 13:56:34.889394 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.889283 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnhdm\" (UniqueName: \"kubernetes.io/projected/1e71a1c4-63a1-482e-bb06-510b2be28dab-kube-api-access-tnhdm\") pod \"perf-node-gather-daemonset-nbzv9\" (UID: \"1e71a1c4-63a1-482e-bb06-510b2be28dab\") " pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" Jul 02 13:56:34.889394 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.889345 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e71a1c4-63a1-482e-bb06-510b2be28dab-sys\") pod \"perf-node-gather-daemonset-nbzv9\" (UID: \"1e71a1c4-63a1-482e-bb06-510b2be28dab\") " pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" Jul 02 13:56:34.889394 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.889376 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e71a1c4-63a1-482e-bb06-510b2be28dab-lib-modules\") pod \"perf-node-gather-daemonset-nbzv9\" (UID: \"1e71a1c4-63a1-482e-bb06-510b2be28dab\") " pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" Jul 02 13:56:34.889711 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.889421 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1e71a1c4-63a1-482e-bb06-510b2be28dab-proc\") pod \"perf-node-gather-daemonset-nbzv9\" (UID: \"1e71a1c4-63a1-482e-bb06-510b2be28dab\") " pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" Jul 02 13:56:34.889711 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.889461 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1e71a1c4-63a1-482e-bb06-510b2be28dab-podres\") pod \"perf-node-gather-daemonset-nbzv9\" (UID: \"1e71a1c4-63a1-482e-bb06-510b2be28dab\") " pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" Jul 02 13:56:34.889711 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.889491 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e71a1c4-63a1-482e-bb06-510b2be28dab-sys\") pod \"perf-node-gather-daemonset-nbzv9\" (UID: \"1e71a1c4-63a1-482e-bb06-510b2be28dab\") " pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" Jul 02 13:56:34.889711 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.889550 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e71a1c4-63a1-482e-bb06-510b2be28dab-lib-modules\") pod \"perf-node-gather-daemonset-nbzv9\" (UID: \"1e71a1c4-63a1-482e-bb06-510b2be28dab\") " pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" Jul 02 13:56:34.889711 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.889600 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1e71a1c4-63a1-482e-bb06-510b2be28dab-proc\") pod \"perf-node-gather-daemonset-nbzv9\" (UID: \"1e71a1c4-63a1-482e-bb06-510b2be28dab\") " pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" Jul 02 13:56:34.889711 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.889625 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1e71a1c4-63a1-482e-bb06-510b2be28dab-podres\") pod \"perf-node-gather-daemonset-nbzv9\" (UID: \"1e71a1c4-63a1-482e-bb06-510b2be28dab\") " pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" Jul 02 13:56:34.907250 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:34.907221 2561 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnhdm\" (UniqueName: \"kubernetes.io/projected/1e71a1c4-63a1-482e-bb06-510b2be28dab-kube-api-access-tnhdm\") pod \"perf-node-gather-daemonset-nbzv9\" (UID: \"1e71a1c4-63a1-482e-bb06-510b2be28dab\") " pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" Jul 02 13:56:35.014325 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:35.014298 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" Jul 02 13:56:35.149405 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:35.149376 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9"] Jul 02 13:56:35.151227 ip-10-0-141-112 kubenswrapper[2561]: W0702 13:56:35.151199 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1e71a1c4_63a1_482e_bb06_510b2be28dab.slice/crio-d61a10c37a73cec0f1653ad2db08362b6621b83852658c60b76e5a7096f2a198 WatchSource:0}: Error finding container d61a10c37a73cec0f1653ad2db08362b6621b83852658c60b76e5a7096f2a198: Status 404 returned error can't find the container with id d61a10c37a73cec0f1653ad2db08362b6621b83852658c60b76e5a7096f2a198 Jul 02 13:56:36.038182 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:36.038140 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" event={"ID":"1e71a1c4-63a1-482e-bb06-510b2be28dab","Type":"ContainerStarted","Data":"e0e5e84fb6d2c71bffcbff2d518dc67de93e82c50a3067efad95e57bfb8fc98d"} Jul 02 13:56:36.038594 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:36.038239 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" event={"ID":"1e71a1c4-63a1-482e-bb06-510b2be28dab","Type":"ContainerStarted","Data":"d61a10c37a73cec0f1653ad2db08362b6621b83852658c60b76e5a7096f2a198"} Jul 02 13:56:36.038594 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:36.038362 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" Jul 02 13:56:36.053606 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:36.053546 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" podStartSLOduration=2.053527546 podStartE2EDuration="2.053527546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 13:56:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 13:56:36.052623123 +0000 UTC m=+902.178759625" watchObservedRunningTime="2026-07-02 13:56:36.053527546 +0000 UTC m=+902.179664048" Jul 02 13:56:36.400474 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:36.400394 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dwvk4_db8817af-4159-4667-b285-8e6325a345c4/dns/0.log" Jul 02 13:56:36.422904 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:36.422877 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dwvk4_db8817af-4159-4667-b285-8e6325a345c4/kube-rbac-proxy/0.log" Jul 02 13:56:36.597105 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:36.597073 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kcdlm_4d160c6f-4496-4913-a38a-26f61cd3effe/dns-node-resolver/0.log" Jul 02 13:56:37.144643 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:37.144605 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cpjcf_1aec8ab3-45e7-422c-985b-0f5541ed6149/node-ca/0.log" Jul 02 13:56:38.170926 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:38.170895 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-685667f9f9-4rcj4_c5698427-1618-4369-9d7c-82c74c7a27d1/kube-auth-proxy/0.log" Jul 02 13:56:38.220015 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:38.219992 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-8x5r9_4e71ad19-1c20-4e48-8c7f-df4c410eb929/payload-pre-processing/0.log" Jul 02 13:56:38.246905 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:38.246877 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-sqv8q_6aefd846-e612-47b2-acf0-98746387507f/payload-processing/2.log" Jul 02 13:56:38.247489 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:38.247470 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-sqv8q_6aefd846-e612-47b2-acf0-98746387507f/payload-processing/1.log" Jul 02 13:56:38.884232 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:38.884198 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dwb5c_a509cdb7-1c6e-4e40-9add-d802b749c4b0/serve-healthcheck-canary/0.log" Jul 02 13:56:39.470877 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:39.470852 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kvknk_0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1/kube-rbac-proxy/0.log" Jul 02 13:56:39.491717 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:39.491686 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kvknk_0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1/exporter/0.log" Jul 02 13:56:39.513174 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:39.513145 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-kvknk_0ec922f5-f2a1-4b62-b95f-a0d0b7f9f0d1/extractor/0.log" Jul 02 13:56:41.797469 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:41.797439 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-86f58868c9-8jw5w_eb695f8e-21bb-4604-a6fa-61e0a1b32150/maas-api/0.log" Jul 02 13:56:41.831255 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:41.831211 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-56c678849-slc8b_ed8b896e-29aa-4383-ae97-57911e4c99ff/manager/0.log" Jul 02 13:56:41.982989 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:41.982954 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5964c6974c-sfjzz_9f18bedb-22f1-43db-be9e-a7012ff27142/manager/0.log" Jul 02 13:56:42.004293 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:42.004261 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5964c6974c-sfjzz_9f18bedb-22f1-43db-be9e-a7012ff27142/copy-manifests/0.log" Jul 02 13:56:42.054841 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:42.054755 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pk25w/perf-node-gather-daemonset-nbzv9" Jul 02 13:56:42.104067 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:42.104038 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-88cc6_f496bfe6-dbd4-405c-90de-7239d985a474/postgres/0.log" Jul 02 13:56:43.506848 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:43.506817 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7497c854c6-tlnxn_2f680c7f-2f26-45f5-ae98-89eb89c8df42/manager/0.log" Jul 02 13:56:43.538390 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:43.538306 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-s8f4q_24d69536-4028-4050-a6a3-5bdf03b57b7f/openshift-lws-operator/0.log" Jul 02 13:56:45.071380 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:45.071269 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-685667f9f9-4rcj4_c5698427-1618-4369-9d7c-82c74c7a27d1/kube-auth-proxy/0.log" Jul 02 13:56:45.100980 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:45.100940 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-8x5r9_4e71ad19-1c20-4e48-8c7f-df4c410eb929/payload-pre-processing/0.log" Jul 02 13:56:45.115294 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:45.115260 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-sqv8q_6aefd846-e612-47b2-acf0-98746387507f/payload-processing/2.log" Jul 02 13:56:45.115887 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:45.115868 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-sqv8q_6aefd846-e612-47b2-acf0-98746387507f/payload-processing/1.log" Jul 02 13:56:46.353326 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:46.353299 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-685667f9f9-4rcj4_c5698427-1618-4369-9d7c-82c74c7a27d1/kube-auth-proxy/0.log" Jul 02 13:56:46.387410 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:46.387382 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-8x5r9_4e71ad19-1c20-4e48-8c7f-df4c410eb929/payload-pre-processing/0.log" Jul 02 13:56:46.403450 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:46.403423 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-sqv8q_6aefd846-e612-47b2-acf0-98746387507f/payload-processing/2.log" Jul 02 13:56:46.403991 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:46.403963 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-sqv8q_6aefd846-e612-47b2-acf0-98746387507f/payload-processing/1.log" Jul 02 13:56:50.474550 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:50.474524 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p66lc_6ac92ae9-c78b-4c00-ae36-41e1c2f5e692/kube-multus-additional-cni-plugins/0.log" Jul 02 13:56:50.495653 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:50.495609 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p66lc_6ac92ae9-c78b-4c00-ae36-41e1c2f5e692/egress-router-binary-copy/0.log" Jul 02 13:56:50.516928 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:50.516896 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p66lc_6ac92ae9-c78b-4c00-ae36-41e1c2f5e692/cni-plugins/0.log" Jul 02 13:56:50.541402 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:50.541374 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p66lc_6ac92ae9-c78b-4c00-ae36-41e1c2f5e692/bond-cni-plugin/0.log" Jul 02 13:56:50.562941 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:50.562912 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p66lc_6ac92ae9-c78b-4c00-ae36-41e1c2f5e692/routeoverride-cni/0.log" Jul 02 13:56:50.584167 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:50.584143 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p66lc_6ac92ae9-c78b-4c00-ae36-41e1c2f5e692/whereabouts-cni-bincopy/0.log" Jul 02 13:56:50.604222 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:50.604195 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p66lc_6ac92ae9-c78b-4c00-ae36-41e1c2f5e692/whereabouts-cni/0.log" Jul 02 13:56:50.815200 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:50.815171 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-xdsnn_bea4fbcc-4063-4f71-a0e5-1e6a7ae875ea/kube-multus/0.log" Jul 02 13:56:50.834766 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:50.834718 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-j9rh2_da0298d6-5c20-459a-864b-0a3146e256ee/network-metrics-daemon/0.log" Jul 02 13:56:50.853597 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:50.853566 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-j9rh2_da0298d6-5c20-459a-864b-0a3146e256ee/kube-rbac-proxy/0.log" Jul 02 13:56:51.990213 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:51.990182 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ppzmg_ddf20f35-9cd8-406c-b529-8e2949a8d4e5/ovn-controller/0.log" Jul 02 13:56:52.011530 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:52.011504 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ppzmg_ddf20f35-9cd8-406c-b529-8e2949a8d4e5/ovn-acl-logging/0.log" Jul 02 13:56:52.036042 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:52.036018 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ppzmg_ddf20f35-9cd8-406c-b529-8e2949a8d4e5/kube-rbac-proxy-node/0.log" Jul 02 13:56:52.056773 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:52.056741 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ppzmg_ddf20f35-9cd8-406c-b529-8e2949a8d4e5/kube-rbac-proxy-ovn-metrics/0.log" Jul 02 13:56:52.078547 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:52.078524 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ppzmg_ddf20f35-9cd8-406c-b529-8e2949a8d4e5/northd/0.log" Jul 02 13:56:52.100400 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:52.100370 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ppzmg_ddf20f35-9cd8-406c-b529-8e2949a8d4e5/nbdb/0.log" Jul 02 13:56:52.123702 ip-10-0-141-112 kubenswrapper[2561]: I0702 13:56:52.123673 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ppzmg_ddf20f35-9cd8-406c-b529-8e2949a8d4e5/sbdb/0.log"