Jul 08 19:03:11.814180 ip-10-0-128-38 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 19:03:11.814191 ip-10-0-128-38 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 19:03:11.814198 ip-10-0-128-38 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 19:03:11.814464 ip-10-0-128-38 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 19:03:21.967214 ip-10-0-128-38 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 08 19:03:21.967232 ip-10-0-128-38 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 7bd8ee7019fa4f5fb8050aa901f3620f -- Jul 08 19:05:25.712806 ip-10-0-128-38 systemd[1]: Starting Kubernetes Kubelet... Jul 08 19:05:26.077680 ip-10-0-128-38 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 19:05:26.077680 ip-10-0-128-38 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 19:05:26.077680 ip-10-0-128-38 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 19:05:26.077680 ip-10-0-128-38 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 19:05:26.077680 ip-10-0-128-38 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 19:05:26.079887 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.079795 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 19:05:26.084940 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084916 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:05:26.084940 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084935 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:05:26.084940 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084939 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:05:26.084940 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084942 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:05:26.084940 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084945 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:05:26.084940 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084948 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:05:26.084940 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084951 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:05:26.085185 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084954 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:05:26.085185 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084957 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:05:26.085185 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084960 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:05:26.085185 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084962 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:05:26.085185 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084965 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:05:26.085185 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084968 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:05:26.085185 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084970 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:05:26.085185 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084973 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:05:26.085185 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084976 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:05:26.085185 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084978 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:05:26.085185 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084981 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:05:26.085185 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084983 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:05:26.085185 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084986 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:05:26.085185 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084989 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:05:26.085185 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084991 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:05:26.085185 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084993 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:05:26.085185 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084996 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:05:26.085185 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.084998 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:05:26.085185 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085001 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:05:26.085185 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085003 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:05:26.085697 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085006 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:05:26.085697 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085010 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:05:26.085697 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085014 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:05:26.085697 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085017 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:05:26.085697 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085023 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:05:26.085697 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085028 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:05:26.085697 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085031 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:05:26.085697 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085033 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:05:26.085697 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085036 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:05:26.085697 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085039 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:05:26.085697 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085042 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:05:26.085697 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085045 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:05:26.085697 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085047 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:05:26.085697 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085050 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:05:26.085697 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085053 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:05:26.085697 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085055 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:05:26.085697 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085058 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:05:26.085697 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085060 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:05:26.086178 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085063 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:05:26.086178 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085065 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:05:26.086178 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085068 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:05:26.086178 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085071 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:05:26.086178 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085073 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:05:26.086178 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085076 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:05:26.086178 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085078 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:05:26.086178 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085081 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:05:26.086178 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085083 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:05:26.086178 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085086 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:05:26.086178 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085088 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:05:26.086178 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085090 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:05:26.086178 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085093 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:05:26.086178 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085096 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:05:26.086178 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085098 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:05:26.086178 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085101 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:05:26.086178 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085103 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:05:26.086178 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085106 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:05:26.086178 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085108 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:05:26.086178 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085110 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:05:26.086685 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085112 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:05:26.086685 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085115 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:05:26.086685 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085117 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:05:26.086685 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085120 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:05:26.086685 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085123 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:05:26.086685 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085125 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:05:26.086685 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085127 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:05:26.086685 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085130 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:05:26.086685 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085134 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:05:26.086685 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085137 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:05:26.086685 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085139 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:05:26.086685 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085142 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:05:26.086685 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085144 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:05:26.086685 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085147 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:05:26.086685 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085149 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:05:26.086685 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085152 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:05:26.086685 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085154 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:05:26.086685 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085156 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:05:26.086685 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085167 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:05:26.086685 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085169 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:05:26.087184 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085172 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:05:26.087184 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085174 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:05:26.087184 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085176 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:05:26.087184 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085179 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:05:26.087184 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085182 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:05:26.087184 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085185 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:05:26.087184 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085187 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:05:26.087184 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085190 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:05:26.087184 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085193 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:05:26.087184 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085195 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:05:26.087184 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085198 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:05:26.087184 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085201 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:05:26.087184 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085203 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:05:26.087184 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085206 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:05:26.087184 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085923 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:05:26.087184 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085931 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:05:26.087184 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085934 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:05:26.087184 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085937 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:05:26.087184 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085940 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:05:26.087184 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085943 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:05:26.087694 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085945 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:05:26.087694 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085948 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:05:26.087694 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085950 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:05:26.087694 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085953 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:05:26.087694 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085955 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:05:26.087694 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085958 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:05:26.087694 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085960 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:05:26.087694 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085962 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:05:26.087694 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085965 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:05:26.087694 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085967 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:05:26.087694 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085970 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:05:26.087694 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085973 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:05:26.087694 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085975 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:05:26.087694 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085978 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:05:26.087694 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085981 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:05:26.087694 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085983 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:05:26.087694 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085986 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:05:26.087694 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085989 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:05:26.087694 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085991 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:05:26.087694 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085994 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:05:26.088189 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085996 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:05:26.088189 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.085998 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:05:26.088189 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086001 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:05:26.088189 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086004 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:05:26.088189 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086006 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:05:26.088189 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086009 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:05:26.088189 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086011 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:05:26.088189 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086014 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:05:26.088189 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086016 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:05:26.088189 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086019 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:05:26.088189 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086022 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:05:26.088189 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086025 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:05:26.088189 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086027 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:05:26.088189 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086030 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:05:26.088189 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086032 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:05:26.088189 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086035 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:05:26.088189 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086037 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:05:26.088189 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086040 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:05:26.088189 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086042 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:05:26.088189 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086044 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:05:26.088702 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086047 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:05:26.088702 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086049 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:05:26.088702 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086051 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:05:26.088702 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086054 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:05:26.088702 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086057 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:05:26.088702 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086060 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:05:26.088702 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086062 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:05:26.088702 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086065 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:05:26.088702 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086067 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:05:26.088702 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086070 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:05:26.088702 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086072 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:05:26.088702 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086075 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:05:26.088702 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086077 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:05:26.088702 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086079 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:05:26.088702 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086082 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:05:26.088702 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086084 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:05:26.088702 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086087 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:05:26.088702 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086089 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:05:26.088702 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086093 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:05:26.089169 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086097 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:05:26.089169 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086100 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:05:26.089169 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086103 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:05:26.089169 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086106 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:05:26.089169 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086109 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:05:26.089169 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086111 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:05:26.089169 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086114 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:05:26.089169 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086117 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:05:26.089169 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086121 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:05:26.089169 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086123 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:05:26.089169 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086126 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:05:26.089169 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086128 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:05:26.089169 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086130 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:05:26.089169 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086133 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:05:26.089169 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086136 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:05:26.089169 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086139 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:05:26.089169 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086142 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:05:26.089169 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086144 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:05:26.089169 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086147 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:05:26.089169 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086150 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:05:26.089677 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086154 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:05:26.089677 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086157 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:05:26.089677 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086160 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:05:26.089677 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086162 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:05:26.089677 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086165 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:05:26.089677 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086168 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:05:26.089677 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086171 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:05:26.089677 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086173 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:05:26.089677 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086176 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:05:26.089677 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086178 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:05:26.089677 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086181 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:05:26.089677 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086184 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:05:26.089677 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086187 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:05:26.089677 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086189 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:05:26.089677 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086601 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:05:26.089677 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086606 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:05:26.089677 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086609 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:05:26.089677 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086611 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:05:26.089677 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086614 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:05:26.090145 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086616 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:05:26.090145 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086619 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:05:26.090145 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086622 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:05:26.090145 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086624 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:05:26.090145 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086627 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:05:26.090145 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086630 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:05:26.090145 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086632 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:05:26.090145 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086635 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:05:26.090145 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086638 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:05:26.090145 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086640 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:05:26.090145 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086642 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:05:26.090145 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086646 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:05:26.090145 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086648 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:05:26.090145 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086651 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:05:26.090145 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086653 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:05:26.090145 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086656 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:05:26.090145 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086658 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:05:26.090145 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086660 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:05:26.090145 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086663 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:05:26.090649 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086665 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:05:26.090649 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086668 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:05:26.090649 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086670 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:05:26.090649 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086672 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:05:26.090649 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086675 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:05:26.090649 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086677 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:05:26.090649 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086680 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:05:26.090649 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086683 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:05:26.090649 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086686 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:05:26.090649 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086689 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:05:26.090649 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086691 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:05:26.090649 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086694 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:05:26.090649 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086696 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:05:26.090649 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086699 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:05:26.090649 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086701 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:05:26.090649 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086704 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:05:26.090649 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086706 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:05:26.090649 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086708 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:05:26.090649 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086711 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:05:26.090649 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086713 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:05:26.091182 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086716 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:05:26.091182 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086718 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:05:26.091182 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086720 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:05:26.091182 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086723 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:05:26.091182 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086725 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:05:26.091182 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086728 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:05:26.091182 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086730 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:05:26.091182 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086733 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:05:26.091182 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086735 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:05:26.091182 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086738 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:05:26.091182 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086740 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:05:26.091182 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086742 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:05:26.091182 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086747 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:05:26.091182 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086751 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:05:26.091182 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086755 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:05:26.091182 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086758 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:05:26.091182 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086761 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:05:26.091182 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086764 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:05:26.091182 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086768 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:05:26.091672 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086772 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:05:26.091672 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086775 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:05:26.091672 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086778 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:05:26.091672 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086780 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:05:26.091672 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086783 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:05:26.091672 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086786 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:05:26.091672 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086788 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:05:26.091672 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086791 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:05:26.091672 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086794 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:05:26.091672 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086796 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:05:26.091672 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086799 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:05:26.091672 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086802 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:05:26.091672 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086804 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:05:26.091672 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086807 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:05:26.091672 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086809 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:05:26.091672 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086811 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:05:26.091672 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086814 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:05:26.091672 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086816 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:05:26.091672 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086818 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:05:26.091672 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086821 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:05:26.092148 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086824 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:05:26.092148 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086826 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:05:26.092148 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086831 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:05:26.092148 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086833 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:05:26.092148 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086836 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:05:26.092148 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086838 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:05:26.092148 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086841 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:05:26.092148 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086844 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:05:26.092148 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086846 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:05:26.092148 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086849 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:05:26.092148 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086851 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:05:26.092148 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086855 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:05:26.092148 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086858 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:05:26.092148 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086860 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:05:26.092148 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086863 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:05:26.092148 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.086865 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:05:26.092148 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.086936 2560 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 19:05:26.092148 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.086944 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 19:05:26.092148 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.086950 2560 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 19:05:26.092148 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.086955 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 19:05:26.092148 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.086960 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.086963 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.086967 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.086971 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.086974 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.086977 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.086980 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.086983 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.086986 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.086989 2560 flags.go:64] FLAG: --cgroup-root="" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.086991 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.086995 2560 flags.go:64] FLAG: --client-ca-file="" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.086999 2560 flags.go:64] FLAG: --cloud-provider="external" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087002 2560 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087007 2560 flags.go:64] FLAG: --cluster-domain="" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087010 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087013 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087016 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087019 2560 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087023 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087026 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087029 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087032 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087036 2560 flags.go:64] FLAG: --contention-profiling="false" Jul 08 19:05:26.092675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087040 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087043 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087045 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087048 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087052 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087055 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087058 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087061 2560 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087063 2560 flags.go:64] FLAG: --enable-server="true" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087066 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087070 2560 flags.go:64] FLAG: --event-burst="100" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087073 2560 flags.go:64] FLAG: --event-qps="50" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087076 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087079 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087081 2560 flags.go:64] FLAG: --eviction-hard="" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087085 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087088 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087091 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087093 2560 flags.go:64] FLAG: --eviction-soft="" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087096 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087099 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087102 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087107 2560 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087109 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087112 2560 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 19:05:26.093280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087114 2560 flags.go:64] FLAG: --feature-gates="" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087118 2560 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087121 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087124 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087127 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087129 2560 flags.go:64] FLAG: --healthz-port="10248" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087134 2560 flags.go:64] FLAG: --help="false" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087136 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-128-38.ec2.internal" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087140 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087143 2560 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087145 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087148 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087152 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087154 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087157 2560 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087159 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087162 2560 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087164 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087167 2560 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087170 2560 flags.go:64] FLAG: --kube-reserved="" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087173 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087175 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087178 2560 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087181 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 19:05:26.093923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087183 2560 flags.go:64] FLAG: --lock-file="" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087186 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087189 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087192 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087197 2560 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087204 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087207 2560 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087210 2560 flags.go:64] FLAG: --logging-format="text" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087213 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087216 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087218 2560 flags.go:64] FLAG: --manifest-url="" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087221 2560 flags.go:64] FLAG: --manifest-url-header="" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087225 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087228 2560 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087233 2560 flags.go:64] FLAG: --max-pods="110" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087236 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087239 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087241 2560 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087244 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087247 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087250 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087252 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087259 2560 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087262 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087265 2560 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 19:05:26.094650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087267 2560 flags.go:64] FLAG: --pod-cidr="" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087270 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087275 2560 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087278 2560 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087281 2560 flags.go:64] FLAG: --pods-per-core="0" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087284 2560 flags.go:64] FLAG: --port="10250" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087287 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087289 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a7a9ab634dc7074a" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087292 2560 flags.go:64] FLAG: --qos-reserved="" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087295 2560 flags.go:64] FLAG: --read-only-port="10255" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087297 2560 flags.go:64] FLAG: --register-node="true" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087300 2560 flags.go:64] FLAG: --register-with-taints="" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087305 2560 flags.go:64] FLAG: --registry-burst="10" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087311 2560 flags.go:64] FLAG: --registry-qps="5" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087314 2560 flags.go:64] FLAG: --reserved-cpus="" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087316 2560 flags.go:64] FLAG: --reserved-memory="" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087320 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087322 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087325 2560 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087328 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087330 2560 flags.go:64] FLAG: --runonce="false" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087333 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087337 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087339 2560 flags.go:64] FLAG: --seccomp-default="false" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087342 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 19:05:26.095263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087345 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087348 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087351 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087353 2560 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087356 2560 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087359 2560 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087361 2560 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087364 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087367 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087370 2560 flags.go:64] FLAG: --system-cgroups="" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087373 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087378 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087380 2560 flags.go:64] FLAG: --tls-cert-file="" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087383 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087386 2560 flags.go:64] FLAG: --tls-min-version="" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087389 2560 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087392 2560 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087394 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087397 2560 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087401 2560 flags.go:64] FLAG: --v="2" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087405 2560 flags.go:64] FLAG: --version="false" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087409 2560 flags.go:64] FLAG: --vmodule="" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087413 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087416 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087558 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:05:26.095890 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087562 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:05:26.096486 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087565 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:05:26.096486 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087568 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:05:26.096486 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087571 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:05:26.096486 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087574 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:05:26.096486 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087577 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:05:26.096486 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087580 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:05:26.096486 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087583 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:05:26.096486 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087585 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:05:26.096486 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087588 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:05:26.096486 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087590 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:05:26.096486 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087593 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:05:26.096486 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087595 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:05:26.096486 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087598 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:05:26.096486 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087601 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:05:26.096486 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087603 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:05:26.096486 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087606 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:05:26.096486 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087608 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:05:26.096486 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087611 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:05:26.096486 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087613 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:05:26.096486 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087615 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:05:26.097057 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087618 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:05:26.097057 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087620 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:05:26.097057 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087623 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:05:26.097057 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087625 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:05:26.097057 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087628 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:05:26.097057 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087631 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:05:26.097057 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087634 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:05:26.097057 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087636 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:05:26.097057 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087640 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:05:26.097057 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087645 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:05:26.097057 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087648 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:05:26.097057 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087651 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:05:26.097057 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087654 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:05:26.097057 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087658 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:05:26.097057 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087661 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:05:26.097057 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087664 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:05:26.097057 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087666 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:05:26.097057 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087669 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:05:26.097057 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087671 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:05:26.097545 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087673 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:05:26.097545 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087676 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:05:26.097545 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087678 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:05:26.097545 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087681 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:05:26.097545 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087683 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:05:26.097545 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087685 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:05:26.097545 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087688 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:05:26.097545 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087690 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:05:26.097545 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087692 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:05:26.097545 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087695 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:05:26.097545 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087697 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:05:26.097545 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087700 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:05:26.097545 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087703 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:05:26.097545 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087706 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:05:26.097545 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087708 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:05:26.097545 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087711 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:05:26.097545 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087713 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:05:26.097545 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087715 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:05:26.097545 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087719 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:05:26.097545 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087721 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:05:26.098044 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087723 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:05:26.098044 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087726 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:05:26.098044 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087729 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:05:26.098044 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087731 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:05:26.098044 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087733 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:05:26.098044 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087736 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:05:26.098044 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087738 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:05:26.098044 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087740 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:05:26.098044 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087744 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:05:26.098044 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087746 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:05:26.098044 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087749 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:05:26.098044 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087751 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:05:26.098044 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087754 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:05:26.098044 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087756 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:05:26.098044 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087759 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:05:26.098044 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087761 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:05:26.098044 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087763 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:05:26.098044 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087766 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:05:26.098044 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087768 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:05:26.098044 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087771 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:05:26.098542 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087773 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:05:26.098542 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087775 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:05:26.098542 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087778 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:05:26.098542 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087780 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:05:26.098542 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087783 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:05:26.098542 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087785 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:05:26.098542 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087788 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:05:26.098542 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087790 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:05:26.098542 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087792 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:05:26.098542 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087795 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:05:26.098542 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087799 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:05:26.098542 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087801 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:05:26.098542 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087803 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:05:26.098542 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087806 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:05:26.098542 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087808 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:05:26.098542 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087811 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:05:26.098542 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087813 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:05:26.098542 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.087816 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:05:26.098991 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.087821 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 19:05:26.098991 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.094490 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 19:05:26.098991 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.094509 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 19:05:26.098991 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.094545 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 19:05:26.098991 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.094552 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 19:05:26.098991 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094610 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:05:26.098991 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094616 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:05:26.098991 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094619 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:05:26.098991 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094622 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:05:26.098991 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094625 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:05:26.098991 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094628 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:05:26.098991 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094630 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:05:26.098991 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094633 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:05:26.098991 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094635 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:05:26.098991 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094638 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:05:26.099375 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094641 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:05:26.099375 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094643 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:05:26.099375 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094646 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:05:26.099375 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094648 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:05:26.099375 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094651 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:05:26.099375 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094654 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:05:26.099375 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094657 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:05:26.099375 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094660 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:05:26.099375 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094662 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:05:26.099375 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094665 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:05:26.099375 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094668 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:05:26.099375 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094671 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:05:26.099375 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094675 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:05:26.099375 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094678 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:05:26.099375 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094681 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:05:26.099375 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094684 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:05:26.099375 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094687 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:05:26.099375 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094690 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:05:26.099375 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094693 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:05:26.099834 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094699 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:05:26.099834 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094702 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:05:26.099834 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094705 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:05:26.099834 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094708 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:05:26.099834 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094710 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:05:26.099834 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094713 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:05:26.099834 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094716 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:05:26.099834 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094719 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:05:26.099834 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094721 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:05:26.099834 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094724 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:05:26.099834 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094726 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:05:26.099834 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094729 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:05:26.099834 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094731 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:05:26.099834 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094734 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:05:26.099834 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094736 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:05:26.099834 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094739 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:05:26.099834 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094741 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:05:26.099834 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094743 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:05:26.099834 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094746 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:05:26.100349 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094749 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:05:26.100349 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094751 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:05:26.100349 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094754 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:05:26.100349 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094756 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:05:26.100349 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094758 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:05:26.100349 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094761 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:05:26.100349 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094763 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:05:26.100349 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094766 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:05:26.100349 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094769 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:05:26.100349 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094771 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:05:26.100349 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094773 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:05:26.100349 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094776 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:05:26.100349 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094779 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:05:26.100349 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094782 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:05:26.100349 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094785 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:05:26.100349 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094788 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:05:26.100349 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094791 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:05:26.100349 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094793 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:05:26.100349 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094796 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:05:26.100349 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094799 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:05:26.100866 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094801 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:05:26.100866 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094804 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:05:26.100866 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094806 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:05:26.100866 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094808 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:05:26.100866 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094811 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:05:26.100866 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094813 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:05:26.100866 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094815 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:05:26.100866 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094818 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:05:26.100866 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094820 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:05:26.100866 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094822 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:05:26.100866 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094825 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:05:26.100866 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094827 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:05:26.100866 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094829 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:05:26.100866 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094832 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:05:26.100866 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094835 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:05:26.100866 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094837 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:05:26.100866 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094840 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:05:26.100866 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094842 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:05:26.100866 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094845 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:05:26.100866 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094847 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:05:26.101346 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094850 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:05:26.101346 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094852 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:05:26.101346 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094854 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:05:26.101346 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094857 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:05:26.101346 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094860 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:05:26.101346 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094862 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:05:26.101346 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094866 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:05:26.101346 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094870 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:05:26.101346 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094873 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:05:26.101346 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094875 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:05:26.101346 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094878 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:05:26.101346 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.094883 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 19:05:26.101346 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094992 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 19:05:26.101346 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.094997 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 19:05:26.101792 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095001 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 19:05:26.101792 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095004 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 19:05:26.101792 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095007 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 19:05:26.101792 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095009 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 19:05:26.101792 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095012 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 19:05:26.101792 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095015 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 19:05:26.101792 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095017 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 19:05:26.101792 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095020 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 08 19:05:26.101792 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095023 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 19:05:26.101792 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095025 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 19:05:26.101792 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095028 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 19:05:26.101792 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095030 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 19:05:26.101792 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095033 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 19:05:26.101792 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095035 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 19:05:26.101792 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095038 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 19:05:26.101792 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095040 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 19:05:26.101792 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095043 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 19:05:26.101792 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095045 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 19:05:26.101792 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095048 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 19:05:26.101792 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095050 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 19:05:26.102262 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095052 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 19:05:26.102262 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095055 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 19:05:26.102262 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095057 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 19:05:26.102262 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095060 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 19:05:26.102262 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095062 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 19:05:26.102262 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095065 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 19:05:26.102262 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095068 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 19:05:26.102262 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095070 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 19:05:26.102262 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095072 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 19:05:26.102262 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095075 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 19:05:26.102262 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095077 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 19:05:26.102262 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095079 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 19:05:26.102262 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095082 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 19:05:26.102262 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095084 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 19:05:26.102262 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095086 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 19:05:26.102262 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095089 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 19:05:26.102262 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095091 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 19:05:26.102262 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095093 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 19:05:26.102262 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095096 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 19:05:26.102740 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095098 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 19:05:26.102740 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095101 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 19:05:26.102740 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095103 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 19:05:26.102740 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095105 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 19:05:26.102740 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095108 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 19:05:26.102740 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095110 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 19:05:26.102740 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095112 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 19:05:26.102740 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095115 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 19:05:26.102740 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095117 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 19:05:26.102740 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095120 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 19:05:26.102740 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095122 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 19:05:26.102740 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095124 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 19:05:26.102740 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095127 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 19:05:26.102740 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095129 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 19:05:26.102740 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095132 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 19:05:26.102740 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095134 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 19:05:26.102740 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095137 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 19:05:26.102740 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095139 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 19:05:26.102740 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095142 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 19:05:26.102740 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095144 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 19:05:26.103234 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095148 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 19:05:26.103234 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095151 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 19:05:26.103234 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095153 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 19:05:26.103234 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095156 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 19:05:26.103234 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095158 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 19:05:26.103234 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095160 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 19:05:26.103234 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095163 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 19:05:26.103234 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095165 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 19:05:26.103234 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095168 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 19:05:26.103234 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095170 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 19:05:26.103234 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095173 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 19:05:26.103234 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095175 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 19:05:26.103234 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095177 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 19:05:26.103234 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095179 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 19:05:26.103234 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095183 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 19:05:26.103234 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095186 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 19:05:26.103234 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095189 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 19:05:26.103234 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095191 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 19:05:26.103234 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095194 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 19:05:26.103857 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095197 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 19:05:26.103857 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095199 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 19:05:26.103857 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095201 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 19:05:26.103857 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095204 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 19:05:26.103857 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095206 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 19:05:26.103857 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095209 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 19:05:26.103857 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095211 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 19:05:26.103857 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095214 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 19:05:26.103857 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095216 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 19:05:26.103857 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095218 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 19:05:26.103857 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095221 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 19:05:26.103857 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095223 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 19:05:26.103857 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095226 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 19:05:26.103857 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095228 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 19:05:26.103857 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095231 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 19:05:26.103857 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095234 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 19:05:26.103857 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095236 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 19:05:26.103857 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095239 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 19:05:26.103857 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:26.095241 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 19:05:26.104564 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.095246 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 19:05:26.104564 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.095357 2560 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 19:05:26.104564 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.098161 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 19:05:26.104564 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.099353 2560 server.go:1019] "Starting client certificate rotation" Jul 08 19:05:26.104564 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.099456 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 19:05:26.104564 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.099496 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 19:05:26.104564 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.102207 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 19:05:26.104564 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.103402 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 19:05:26.114193 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.114176 2560 log.go:25] "Validated CRI v1 runtime API" Jul 08 19:05:26.116154 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.116140 2560 log.go:25] "Validated CRI v1 image API" Jul 08 19:05:26.117092 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.117074 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 19:05:26.121036 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.121009 2560 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 cc03c6d9-6fa6-4169-964e-8da0d6ee3e15:/dev/nvme0n1p4 d97cdad3-1a7c-4db8-b0a0-f3ac793e6ee6:/dev/nvme0n1p3] Jul 08 19:05:26.121132 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.121036 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 19:05:26.126280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.126166 2560 manager.go:216] Machine: {Timestamp:2026-07-08 19:05:26.124958706 +0000 UTC m=+0.303092704 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100629 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2fc5932cd09f2079d6e79af410fae1 SystemUUID:ec2fc593-2cd0-9f20-79d6-e79af410fae1 BootID:7bd8ee70-19fa-4f5f-b805-0aa901f3620f Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:50:3d:b7:e4:23 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:50:3d:b7:e4:23 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:ab:df:03:4c:b9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 19:05:26.126280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.126270 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 19:05:26.126413 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.126380 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 19:05:26.129734 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.129708 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 19:05:26.129887 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.129738 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-38.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 19:05:26.130625 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.130615 2560 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 19:05:26.130666 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.130627 2560 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 19:05:26.130666 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.130635 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 19:05:26.131298 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.131288 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 19:05:26.131449 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.131435 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 19:05:26.132571 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.132559 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 19:05:26.135312 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.135300 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 08 19:05:26.135437 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.135427 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 19:05:26.135475 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.135463 2560 kubelet.go:486] "Attempting to sync node with API server" Jul 08 19:05:26.135503 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.135479 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 19:05:26.135503 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.135498 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 19:05:26.135568 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.135511 2560 kubelet.go:398] "Adding apiserver pod source" Jul 08 19:05:26.135568 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.135552 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 19:05:26.135943 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.135931 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 19:05:26.135986 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.135949 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 19:05:26.138127 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.138112 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 19:05:26.139785 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.139771 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 19:05:26.139829 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.139804 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 19:05:26.140849 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.140837 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 19:05:26.140883 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.140853 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 19:05:26.140883 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.140859 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 19:05:26.140883 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.140865 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 19:05:26.140883 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.140874 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 19:05:26.140883 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.140880 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 19:05:26.141014 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.140888 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 19:05:26.141014 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.140894 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 19:05:26.141014 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.140900 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 19:05:26.141014 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.140906 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 19:05:26.141014 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.140922 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 19:05:26.141014 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.140928 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 19:05:26.141666 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.141656 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 19:05:26.141666 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.141666 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 19:05:26.145490 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.145464 2560 server.go:1268] "Started kubelet" Jul 08 19:05:26.148830 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.145692 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 19:05:26.148949 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.148892 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-38.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 19:05:26.149006 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.148949 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 19:05:26.149054 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.149033 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 19:05:26.149054 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:26.149035 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-38.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 19:05:26.149380 ip-10-0-128-38 systemd[1]: Started Kubernetes Kubelet. Jul 08 19:05:26.151799 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:26.150162 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 19:05:26.151799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.150217 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 19:05:26.151799 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:26.150823 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 19:05:26.153443 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.153415 2560 server.go:310] "Adding debug handlers to kubelet server" Jul 08 19:05:26.153626 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:26.152428 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-38.ec2.internal.18c065ee317a120a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-38.ec2.internal,UID:ip-10-0-128-38.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-38.ec2.internal,},FirstTimestamp:2026-07-08 19:05:26.145421834 +0000 UTC m=+0.323555830,LastTimestamp:2026-07-08 19:05:26.145421834 +0000 UTC m=+0.323555830,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-38.ec2.internal,}" Jul 08 19:05:26.155863 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.155850 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 19:05:26.155925 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.155866 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 19:05:26.156024 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:26.156005 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 08 19:05:26.156417 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.156397 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 19:05:26.156494 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.156397 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 19:05:26.156494 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.156422 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 19:05:26.156606 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.156547 2560 reconstruct.go:97] "Volume reconstruction finished" Jul 08 19:05:26.156606 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.156557 2560 reconciler.go:29] "Reconciler: start to sync state" Jul 08 19:05:26.156703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.156648 2560 factory.go:55] Registering systemd factory Jul 08 19:05:26.156703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.156662 2560 factory.go:223] Registration of the systemd container factory successfully Jul 08 19:05:26.157030 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.157008 2560 factory.go:153] Registering CRI-O factory Jul 08 19:05:26.157030 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.157029 2560 factory.go:223] Registration of the crio container factory successfully Jul 08 19:05:26.157119 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.157092 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 19:05:26.157158 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.157119 2560 factory.go:103] Registering Raw factory Jul 08 19:05:26.157158 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.157135 2560 manager.go:1196] Started watching for new ooms in manager Jul 08 19:05:26.157627 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.157606 2560 manager.go:349] Starting recovery of all containers Jul 08 19:05:26.158183 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:26.158159 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 08 19:05:26.158279 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:26.158260 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-38.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 08 19:05:26.159072 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.159047 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c4l67" Jul 08 19:05:26.164741 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.164556 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 19:05:26.167612 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.167591 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c4l67" Jul 08 19:05:26.168265 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.168244 2560 manager.go:354] Recovery completed Jul 08 19:05:26.173433 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.173418 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:05:26.175903 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.175885 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:05:26.175989 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.175921 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:05:26.175989 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.175933 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:05:26.176557 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.176539 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 19:05:26.176557 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.176556 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 19:05:26.176672 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.176565 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:05:26.176672 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.176577 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 08 19:05:26.177640 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.177621 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:05:26.177741 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.177652 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:05:26.177741 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.177667 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:05:26.177741 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.177687 2560 policy_none.go:49] "None policy: Start" Jul 08 19:05:26.177741 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.177705 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 19:05:26.177741 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.177718 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 19:05:26.179075 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.179060 2560 policy_none.go:47] "Start" Jul 08 19:05:26.214121 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.214095 2560 manager.go:337] "Starting Device Plugin manager" Jul 08 19:05:26.214244 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:26.214130 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 19:05:26.214244 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.214142 2560 server.go:87] "Starting device plugin registration server" Jul 08 19:05:26.214418 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.214400 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 19:05:26.214542 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.214414 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 19:05:26.214601 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.214546 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 19:05:26.214668 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.214638 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 19:05:26.214668 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.214648 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 19:05:26.215203 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:26.215184 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 19:05:26.215295 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:26.215237 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-38.ec2.internal\" not found" Jul 08 19:05:26.314959 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.314933 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:05:26.316107 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.316049 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:05:26.316107 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.316085 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:05:26.316107 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.316102 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:05:26.316241 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.316136 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-38.ec2.internal" Jul 08 19:05:26.329165 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.329107 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-38.ec2.internal" Jul 08 19:05:26.329165 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:26.329133 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-38.ec2.internal\": node \"ip-10-0-128-38.ec2.internal\" not found" Jul 08 19:05:26.346792 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:26.346763 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 08 19:05:26.348325 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.348308 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 19:05:26.348377 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.348347 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 19:05:26.348377 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.348367 2560 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 19:05:26.348441 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:26.348396 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 19:05:26.350735 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.350717 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:05:26.447696 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:26.447660 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 08 19:05:26.448752 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.448731 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal"] Jul 08 19:05:26.448813 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.448804 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:05:26.450771 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.450754 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:05:26.450868 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.450783 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:05:26.450868 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.450793 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:05:26.452249 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.452237 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:05:26.452361 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.452346 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" Jul 08 19:05:26.452404 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.452377 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:05:26.453069 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.453054 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:05:26.453069 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.453061 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:05:26.453190 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.453079 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:05:26.453190 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.453086 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:05:26.453190 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.453089 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:05:26.453190 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.453096 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:05:26.454332 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.454319 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal" Jul 08 19:05:26.454386 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.454343 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 19:05:26.455051 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.455033 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientMemory" Jul 08 19:05:26.455119 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.455067 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 19:05:26.455119 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.455081 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeHasSufficientPID" Jul 08 19:05:26.481353 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:26.481326 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-38.ec2.internal\" not found" node="ip-10-0-128-38.ec2.internal" Jul 08 19:05:26.485931 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:26.485914 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-38.ec2.internal\" not found" node="ip-10-0-128-38.ec2.internal" Jul 08 19:05:26.547973 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:26.547936 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 08 19:05:26.558252 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.558222 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bfc2ad129405fbfb160c1db3475eb64b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal\" (UID: \"bfc2ad129405fbfb160c1db3475eb64b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" Jul 08 19:05:26.558383 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.558256 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/301ed78f06a8f545f44b18fb2b42ca58-config\") pod \"kube-apiserver-proxy-ip-10-0-128-38.ec2.internal\" (UID: \"301ed78f06a8f545f44b18fb2b42ca58\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal" Jul 08 19:05:26.558383 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.558281 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bfc2ad129405fbfb160c1db3475eb64b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal\" (UID: \"bfc2ad129405fbfb160c1db3475eb64b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" Jul 08 19:05:26.649085 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:26.649005 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 08 19:05:26.659446 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.659417 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bfc2ad129405fbfb160c1db3475eb64b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal\" (UID: \"bfc2ad129405fbfb160c1db3475eb64b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" Jul 08 19:05:26.659511 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.659452 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/301ed78f06a8f545f44b18fb2b42ca58-config\") pod \"kube-apiserver-proxy-ip-10-0-128-38.ec2.internal\" (UID: \"301ed78f06a8f545f44b18fb2b42ca58\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal" Jul 08 19:05:26.659511 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.659471 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bfc2ad129405fbfb160c1db3475eb64b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal\" (UID: \"bfc2ad129405fbfb160c1db3475eb64b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" Jul 08 19:05:26.659634 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.659522 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bfc2ad129405fbfb160c1db3475eb64b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal\" (UID: \"bfc2ad129405fbfb160c1db3475eb64b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" Jul 08 19:05:26.659634 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.659544 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bfc2ad129405fbfb160c1db3475eb64b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal\" (UID: \"bfc2ad129405fbfb160c1db3475eb64b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" Jul 08 19:05:26.659634 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.659573 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/301ed78f06a8f545f44b18fb2b42ca58-config\") pod \"kube-apiserver-proxy-ip-10-0-128-38.ec2.internal\" (UID: \"301ed78f06a8f545f44b18fb2b42ca58\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal" Jul 08 19:05:26.749619 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:26.749576 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 08 19:05:26.784033 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.784004 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" Jul 08 19:05:26.788891 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:26.788875 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal" Jul 08 19:05:26.850115 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:26.850079 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 08 19:05:26.950488 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:26.950433 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 08 19:05:27.054861 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:27.054811 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 08 19:05:27.098615 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:27.098578 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 19:05:27.099168 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:27.098760 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 19:05:27.155035 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:27.154942 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-38.ec2.internal\" not found" Jul 08 19:05:27.172818 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:27.172768 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 19:00:26 +0000 UTC" deadline="2028-03-24 03:58:34.253348032 +0000 UTC" Jul 08 19:05:27.172818 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:27.172812 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14984h53m7.080541698s" Jul 08 19:05:27.224759 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:27.224725 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:05:27.256988 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:27.256959 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" Jul 08 19:05:27.269882 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:27.269849 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 19:05:27.271578 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:27.271560 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal" Jul 08 19:05:27.280364 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:27.280338 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 19:05:27.387453 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:27.387421 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod301ed78f06a8f545f44b18fb2b42ca58.slice/crio-784a1db3d746d433d900e6a73a0781221a5da5bef6cd42e5092f5be7d426ab01 WatchSource:0}: Error finding container 784a1db3d746d433d900e6a73a0781221a5da5bef6cd42e5092f5be7d426ab01: Status 404 returned error can't find the container with id 784a1db3d746d433d900e6a73a0781221a5da5bef6cd42e5092f5be7d426ab01 Jul 08 19:05:27.391748 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:27.391733 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:05:27.429566 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:27.429539 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfc2ad129405fbfb160c1db3475eb64b.slice/crio-ba3c85ac3f1f69c58f5319cc65a7fb5385424f14764c22f93b4513a5c563a7e6 WatchSource:0}: Error finding container ba3c85ac3f1f69c58f5319cc65a7fb5385424f14764c22f93b4513a5c563a7e6: Status 404 returned error can't find the container with id ba3c85ac3f1f69c58f5319cc65a7fb5385424f14764c22f93b4513a5c563a7e6 Jul 08 19:05:27.501697 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:27.501658 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:05:27.522412 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:27.522386 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:05:28.136342 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.136308 2560 apiserver.go:52] "Watching apiserver" Jul 08 19:05:28.147373 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.147335 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 19:05:28.147824 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.147797 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal","openshift-network-diagnostics/network-check-target-qm8rp","openshift-network-operator/iptables-alerter-f8nv2","openshift-ovn-kubernetes/ovnkube-node-nklcn","kube-system/global-pull-secret-syncer-fmbfk","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5","openshift-cluster-node-tuning-operator/tuned-pvrsj","openshift-dns/node-resolver-sg8ph","openshift-image-registry/node-ca-xs94k","openshift-multus/multus-5sxcv","openshift-multus/multus-additional-cni-plugins-5wxhc","openshift-multus/network-metrics-daemon-2bqn6","kube-system/konnectivity-agent-wndp8","kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal"] Jul 08 19:05:28.149629 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.149600 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.150673 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.150653 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wndp8" Jul 08 19:05:28.151794 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.151774 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f8nv2" Jul 08 19:05:28.152335 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.152316 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:05:28.152570 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.152553 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 19:05:28.152707 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.152682 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-sw7t4\"" Jul 08 19:05:28.153060 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.153033 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.153160 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.153108 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 19:05:28.153378 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.153357 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 19:05:28.153900 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.153882 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-v6c9m\"" Jul 08 19:05:28.154189 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.154172 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 19:05:28.154423 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.154406 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:05:28.154784 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.154764 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 19:05:28.155081 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.155064 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:28.155157 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:28.155133 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fmbfk" podUID="77822693-2416-4400-87cf-4619ac479e79" Jul 08 19:05:28.155213 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.155143 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tt58w\"" Jul 08 19:05:28.155633 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.155375 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 19:05:28.155633 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.155558 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 19:05:28.155875 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.155856 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 19:05:28.156409 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.156127 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 19:05:28.156409 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.156168 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 19:05:28.156409 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.156127 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 19:05:28.156678 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.156661 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-z7dcr\"" Jul 08 19:05:28.156994 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.156974 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 19:05:28.157084 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.157033 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.157573 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.157555 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xs94k" Jul 08 19:05:28.159335 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.159317 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sg8ph" Jul 08 19:05:28.159769 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.159747 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 19:05:28.159935 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.159828 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 19:05:28.159935 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.159852 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 19:05:28.160131 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.159984 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rb4bk\"" Jul 08 19:05:28.160131 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.160076 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 19:05:28.160273 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.160143 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-6g28z\"" Jul 08 19:05:28.160273 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.160228 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 19:05:28.160417 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.160373 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 19:05:28.160995 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.160973 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:28.161090 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:28.161070 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qm8rp" podUID="03cbe6c1-781c-4091-86ca-9de8256eda85" Jul 08 19:05:28.161672 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.161651 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 19:05:28.162101 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.162078 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-k5nx8\"" Jul 08 19:05:28.162188 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.162135 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 19:05:28.162472 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.162445 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.164614 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.164593 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.166401 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.166379 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 19:05:28.166674 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.166659 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 19:05:28.167703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.166912 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 19:05:28.167703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167103 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-cphpk\"" Jul 08 19:05:28.167703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167167 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 19:05:28.167703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167198 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 19:05:28.167703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167367 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zkh7\" (UniqueName: \"kubernetes.io/projected/04ab4794-25e2-4c82-9e44-b460a3993f26-kube-api-access-9zkh7\") pod \"node-ca-xs94k\" (UID: \"04ab4794-25e2-4c82-9e44-b460a3993f26\") " pod="openshift-image-registry/node-ca-xs94k" Jul 08 19:05:28.167703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167396 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-etc-systemd\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.167703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167413 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-host\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.167703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167434 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/69978d76-a5ad-4ebf-9d22-8335ec387429-konnectivity-ca\") pod \"konnectivity-agent-wndp8\" (UID: \"69978d76-a5ad-4ebf-9d22-8335ec387429\") " pod="kube-system/konnectivity-agent-wndp8" Jul 08 19:05:28.167703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167460 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b439b91-714c-4386-a270-fbb3115347c6-host-slash\") pod \"iptables-alerter-f8nv2\" (UID: \"2b439b91-714c-4386-a270-fbb3115347c6\") " pod="openshift-network-operator/iptables-alerter-f8nv2" Jul 08 19:05:28.167703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167484 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/426767c6-42b0-4249-b95c-d0487aa20faa-env-overrides\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.167703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167507 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/45b5625b-d7f7-488f-ae0c-c8dd8103c4f9-registration-dir\") pod \"aws-ebs-csi-driver-node-bzdv5\" (UID: \"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.167703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167550 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-etc-modprobe-d\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.167703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167595 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-run-systemd\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.167703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167625 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-run-openvswitch\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.167703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167651 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret\") pod \"global-pull-secret-syncer-fmbfk\" (UID: \"77822693-2416-4400-87cf-4619ac479e79\") " pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:28.167703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167677 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/45b5625b-d7f7-488f-ae0c-c8dd8103c4f9-device-dir\") pod \"aws-ebs-csi-driver-node-bzdv5\" (UID: \"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.167703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167694 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-etc-sysctl-d\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.167703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167710 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-sys\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.167703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167733 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-systemd-units\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.168799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167755 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-etc-openvswitch\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.168799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167798 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-run-ovn\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.168799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167832 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-host-cni-netd\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.168799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167851 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/426767c6-42b0-4249-b95c-d0487aa20faa-ovnkube-script-lib\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.168799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167877 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4d42\" (UniqueName: \"kubernetes.io/projected/45b5625b-d7f7-488f-ae0c-c8dd8103c4f9-kube-api-access-x4d42\") pod \"aws-ebs-csi-driver-node-bzdv5\" (UID: \"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.168799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167893 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-run\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.168799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167908 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-host-run-netns\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.168799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167946 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-var-lib-openvswitch\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.168799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167973 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-node-log\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.168799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.167990 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-host-cni-bin\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.168799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168005 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/426767c6-42b0-4249-b95c-d0487aa20faa-ovnkube-config\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.168799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168037 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/77822693-2416-4400-87cf-4619ac479e79-kubelet-config\") pod \"global-pull-secret-syncer-fmbfk\" (UID: \"77822693-2416-4400-87cf-4619ac479e79\") " pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:28.168799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168056 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/45b5625b-d7f7-488f-ae0c-c8dd8103c4f9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bzdv5\" (UID: \"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.168799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168073 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-etc-sysconfig\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.168799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168086 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fa2c98dd-2141-482f-b02f-9e522908fd71-etc-tuned\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.168799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168112 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r97p\" (UniqueName: \"kubernetes.io/projected/2b439b91-714c-4386-a270-fbb3115347c6-kube-api-access-6r97p\") pod \"iptables-alerter-f8nv2\" (UID: \"2b439b91-714c-4386-a270-fbb3115347c6\") " pod="openshift-network-operator/iptables-alerter-f8nv2" Jul 08 19:05:28.169459 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168128 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-host-kubelet\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.169459 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168144 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/45b5625b-d7f7-488f-ae0c-c8dd8103c4f9-socket-dir\") pod \"aws-ebs-csi-driver-node-bzdv5\" (UID: \"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.169459 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168159 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/45b5625b-d7f7-488f-ae0c-c8dd8103c4f9-etc-selinux\") pod \"aws-ebs-csi-driver-node-bzdv5\" (UID: \"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.169459 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168173 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa2c98dd-2141-482f-b02f-9e522908fd71-tmp\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.169459 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168189 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-host-slash\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.169459 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168205 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-log-socket\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.169459 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168225 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-host-run-ovn-kubernetes\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.169459 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168260 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.169459 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168299 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/45b5625b-d7f7-488f-ae0c-c8dd8103c4f9-sys-fs\") pod \"aws-ebs-csi-driver-node-bzdv5\" (UID: \"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.169459 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168321 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/04ab4794-25e2-4c82-9e44-b460a3993f26-host\") pod \"node-ca-xs94k\" (UID: \"04ab4794-25e2-4c82-9e44-b460a3993f26\") " pod="openshift-image-registry/node-ca-xs94k" Jul 08 19:05:28.169459 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168345 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-etc-kubernetes\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.169459 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168370 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdvd8\" (UniqueName: \"kubernetes.io/projected/fa2c98dd-2141-482f-b02f-9e522908fd71-kube-api-access-qdvd8\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.169459 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168435 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b439b91-714c-4386-a270-fbb3115347c6-iptables-alerter-script\") pod \"iptables-alerter-f8nv2\" (UID: \"2b439b91-714c-4386-a270-fbb3115347c6\") " pod="openshift-network-operator/iptables-alerter-f8nv2" Jul 08 19:05:28.169459 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168485 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/426767c6-42b0-4249-b95c-d0487aa20faa-ovn-node-metrics-cert\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.169459 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168510 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bpc5\" (UniqueName: \"kubernetes.io/projected/426767c6-42b0-4249-b95c-d0487aa20faa-kube-api-access-2bpc5\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.169459 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168534 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/77822693-2416-4400-87cf-4619ac479e79-dbus\") pod \"global-pull-secret-syncer-fmbfk\" (UID: \"77822693-2416-4400-87cf-4619ac479e79\") " pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:28.170206 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168573 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-etc-sysctl-conf\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.170206 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168595 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-lib-modules\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.170206 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168619 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-var-lib-kubelet\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.170206 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168644 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/69978d76-a5ad-4ebf-9d22-8335ec387429-agent-certs\") pod \"konnectivity-agent-wndp8\" (UID: \"69978d76-a5ad-4ebf-9d22-8335ec387429\") " pod="kube-system/konnectivity-agent-wndp8" Jul 08 19:05:28.170206 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.168669 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/04ab4794-25e2-4c82-9e44-b460a3993f26-serviceca\") pod \"node-ca-xs94k\" (UID: \"04ab4794-25e2-4c82-9e44-b460a3993f26\") " pod="openshift-image-registry/node-ca-xs94k" Jul 08 19:05:28.171553 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.170648 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:28.171553 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:28.170715 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bqn6" podUID="3981d8c6-c84c-439a-bf4b-d9834812c8bb" Jul 08 19:05:28.171553 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.170829 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-z47mx\"" Jul 08 19:05:28.171553 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.171409 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 19:05:28.175307 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.175274 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 19:05:28.193148 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.193122 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-l46d4" Jul 08 19:05:28.203159 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.203134 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-l46d4" Jul 08 19:05:28.257700 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.257671 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 19:05:28.269325 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.269286 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa2c98dd-2141-482f-b02f-9e522908fd71-tmp\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.269505 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.269334 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-host-slash\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.269505 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.269355 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.269505 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.269386 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-multus-cni-dir\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.269505 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.269418 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-multus-conf-dir\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.269505 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.269438 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-host-slash\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.269505 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.269444 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs\") pod \"network-metrics-daemon-2bqn6\" (UID: \"3981d8c6-c84c-439a-bf4b-d9834812c8bb\") " pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:28.269505 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.269452 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.269505 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.269477 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6349a4f6-297b-4718-9981-b1dbc8415f79-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.269916 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.269562 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-etc-kubernetes\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.269916 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.269596 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdvd8\" (UniqueName: \"kubernetes.io/projected/fa2c98dd-2141-482f-b02f-9e522908fd71-kube-api-access-qdvd8\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.269916 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.269678 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/426767c6-42b0-4249-b95c-d0487aa20faa-ovn-node-metrics-cert\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.269916 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.269710 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-etc-kubernetes\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.269916 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.269757 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 19:05:28.269916 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.269762 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/77822693-2416-4400-87cf-4619ac479e79-dbus\") pod \"global-pull-secret-syncer-fmbfk\" (UID: \"77822693-2416-4400-87cf-4619ac479e79\") " pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:28.269916 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.269909 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7d66e59-7d94-4a00-86f4-fbb8dfe65280-tmp-dir\") pod \"node-resolver-sg8ph\" (UID: \"a7d66e59-7d94-4a00-86f4-fbb8dfe65280\") " pod="openshift-dns/node-resolver-sg8ph" Jul 08 19:05:28.270280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.269915 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/77822693-2416-4400-87cf-4619ac479e79-dbus\") pod \"global-pull-secret-syncer-fmbfk\" (UID: \"77822693-2416-4400-87cf-4619ac479e79\") " pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:28.270280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.269946 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-etc-sysctl-conf\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.270280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270037 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/04ab4794-25e2-4c82-9e44-b460a3993f26-serviceca\") pod \"node-ca-xs94k\" (UID: \"04ab4794-25e2-4c82-9e44-b460a3993f26\") " pod="openshift-image-registry/node-ca-xs94k" Jul 08 19:05:28.270280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270074 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-host-var-lib-cni-bin\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.270280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270131 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/69978d76-a5ad-4ebf-9d22-8335ec387429-konnectivity-ca\") pod \"konnectivity-agent-wndp8\" (UID: \"69978d76-a5ad-4ebf-9d22-8335ec387429\") " pod="kube-system/konnectivity-agent-wndp8" Jul 08 19:05:28.270280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270140 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-etc-sysctl-conf\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.270280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270157 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b439b91-714c-4386-a270-fbb3115347c6-host-slash\") pod \"iptables-alerter-f8nv2\" (UID: \"2b439b91-714c-4386-a270-fbb3115347c6\") " pod="openshift-network-operator/iptables-alerter-f8nv2" Jul 08 19:05:28.270280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270197 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/426767c6-42b0-4249-b95c-d0487aa20faa-env-overrides\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.270280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270220 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-etc-modprobe-d\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.270280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270245 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-run-systemd\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.270280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270232 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b439b91-714c-4386-a270-fbb3115347c6-host-slash\") pod \"iptables-alerter-f8nv2\" (UID: \"2b439b91-714c-4386-a270-fbb3115347c6\") " pod="openshift-network-operator/iptables-alerter-f8nv2" Jul 08 19:05:28.270280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270266 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret\") pod \"global-pull-secret-syncer-fmbfk\" (UID: \"77822693-2416-4400-87cf-4619ac479e79\") " pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:28.270910 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270302 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-etc-kubernetes\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.270910 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270320 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-run-systemd\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.270910 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270355 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6349a4f6-297b-4718-9981-b1dbc8415f79-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.270910 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:28.270363 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:05:28.270910 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270377 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nl2d\" (UniqueName: \"kubernetes.io/projected/6349a4f6-297b-4718-9981-b1dbc8415f79-kube-api-access-8nl2d\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.270910 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270397 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-sys\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.270910 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270414 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-etc-openvswitch\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.270910 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270431 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-host-cni-netd\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.270910 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:28.270453 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret podName:77822693-2416-4400-87cf-4619ac479e79 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:28.770428711 +0000 UTC m=+2.948562712 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret") pod "global-pull-secret-syncer-fmbfk" (UID: "77822693-2416-4400-87cf-4619ac479e79") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:05:28.270910 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270458 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-host-cni-netd\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.270910 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270490 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/04ab4794-25e2-4c82-9e44-b460a3993f26-host\") pod \"node-ca-xs94k\" (UID: \"04ab4794-25e2-4c82-9e44-b460a3993f26\") " pod="openshift-image-registry/node-ca-xs94k" Jul 08 19:05:28.270910 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270516 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-sys\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.270910 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270521 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-run\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.270910 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270571 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-host-run-netns\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.270910 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270543 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-etc-openvswitch\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.270910 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270615 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-node-log\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.270910 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270645 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-host-cni-bin\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.271724 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270673 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-host-run-netns\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.271724 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270689 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-etc-modprobe-d\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.271724 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270703 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-host-var-lib-cni-multus\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.271724 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270731 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-etc-sysconfig\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.271724 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270764 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/04ab4794-25e2-4c82-9e44-b460a3993f26-serviceca\") pod \"node-ca-xs94k\" (UID: \"04ab4794-25e2-4c82-9e44-b460a3993f26\") " pod="openshift-image-registry/node-ca-xs94k" Jul 08 19:05:28.271724 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270782 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r97p\" (UniqueName: \"kubernetes.io/projected/2b439b91-714c-4386-a270-fbb3115347c6-kube-api-access-6r97p\") pod \"iptables-alerter-f8nv2\" (UID: \"2b439b91-714c-4386-a270-fbb3115347c6\") " pod="openshift-network-operator/iptables-alerter-f8nv2" Jul 08 19:05:28.271724 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270812 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-host-kubelet\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.271724 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270826 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/69978d76-a5ad-4ebf-9d22-8335ec387429-konnectivity-ca\") pod \"konnectivity-agent-wndp8\" (UID: \"69978d76-a5ad-4ebf-9d22-8335ec387429\") " pod="kube-system/konnectivity-agent-wndp8" Jul 08 19:05:28.271724 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270840 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/45b5625b-d7f7-488f-ae0c-c8dd8103c4f9-socket-dir\") pod \"aws-ebs-csi-driver-node-bzdv5\" (UID: \"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.271724 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270847 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-node-log\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.271724 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270857 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/04ab4794-25e2-4c82-9e44-b460a3993f26-host\") pod \"node-ca-xs94k\" (UID: \"04ab4794-25e2-4c82-9e44-b460a3993f26\") " pod="openshift-image-registry/node-ca-xs94k" Jul 08 19:05:28.271724 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270869 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-hostroot\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.271724 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270893 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-host-cni-bin\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.271724 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270896 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6349a4f6-297b-4718-9981-b1dbc8415f79-os-release\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.271724 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270902 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-host-run-netns\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.271724 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270921 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-host-kubelet\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.271724 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270923 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-log-socket\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.271724 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270962 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-etc-sysconfig\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.272856 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270972 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-host-run-ovn-kubernetes\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.272856 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270990 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/426767c6-42b0-4249-b95c-d0487aa20faa-env-overrides\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.272856 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271007 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/45b5625b-d7f7-488f-ae0c-c8dd8103c4f9-sys-fs\") pod \"aws-ebs-csi-driver-node-bzdv5\" (UID: \"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.272856 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271014 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/45b5625b-d7f7-488f-ae0c-c8dd8103c4f9-socket-dir\") pod \"aws-ebs-csi-driver-node-bzdv5\" (UID: \"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.272856 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271026 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-host-run-ovn-kubernetes\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.272856 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.270899 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-run\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.272856 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271045 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-multus-socket-dir-parent\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.272856 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271054 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/45b5625b-d7f7-488f-ae0c-c8dd8103c4f9-sys-fs\") pod \"aws-ebs-csi-driver-node-bzdv5\" (UID: \"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.272856 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271077 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-log-socket\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.272856 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271070 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-multus-daemon-config\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.272856 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271113 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a7d66e59-7d94-4a00-86f4-fbb8dfe65280-hosts-file\") pod \"node-resolver-sg8ph\" (UID: \"a7d66e59-7d94-4a00-86f4-fbb8dfe65280\") " pod="openshift-dns/node-resolver-sg8ph" Jul 08 19:05:28.272856 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271136 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6349a4f6-297b-4718-9981-b1dbc8415f79-cnibin\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.272856 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271186 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b439b91-714c-4386-a270-fbb3115347c6-iptables-alerter-script\") pod \"iptables-alerter-f8nv2\" (UID: \"2b439b91-714c-4386-a270-fbb3115347c6\") " pod="openshift-network-operator/iptables-alerter-f8nv2" Jul 08 19:05:28.272856 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271218 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bpc5\" (UniqueName: \"kubernetes.io/projected/426767c6-42b0-4249-b95c-d0487aa20faa-kube-api-access-2bpc5\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.272856 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271248 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-lib-modules\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.272856 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271378 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-lib-modules\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.272856 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271418 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-var-lib-kubelet\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.273766 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271452 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/69978d76-a5ad-4ebf-9d22-8335ec387429-agent-certs\") pod \"konnectivity-agent-wndp8\" (UID: \"69978d76-a5ad-4ebf-9d22-8335ec387429\") " pod="kube-system/konnectivity-agent-wndp8" Jul 08 19:05:28.273766 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271483 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zkh7\" (UniqueName: \"kubernetes.io/projected/04ab4794-25e2-4c82-9e44-b460a3993f26-kube-api-access-9zkh7\") pod \"node-ca-xs94k\" (UID: \"04ab4794-25e2-4c82-9e44-b460a3993f26\") " pod="openshift-image-registry/node-ca-xs94k" Jul 08 19:05:28.273766 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271509 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-etc-systemd\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.273766 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271557 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-host\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.273766 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271561 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-var-lib-kubelet\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.273766 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271613 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-host\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.273766 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271627 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-etc-systemd\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.273766 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271642 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/45b5625b-d7f7-488f-ae0c-c8dd8103c4f9-registration-dir\") pod \"aws-ebs-csi-driver-node-bzdv5\" (UID: \"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.273766 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271671 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-cnibin\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.273766 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271684 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/45b5625b-d7f7-488f-ae0c-c8dd8103c4f9-registration-dir\") pod \"aws-ebs-csi-driver-node-bzdv5\" (UID: \"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.273766 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271697 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-host-run-k8s-cni-cncf-io\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.273766 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271725 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnf7n\" (UniqueName: \"kubernetes.io/projected/3981d8c6-c84c-439a-bf4b-d9834812c8bb-kube-api-access-nnf7n\") pod \"network-metrics-daemon-2bqn6\" (UID: \"3981d8c6-c84c-439a-bf4b-d9834812c8bb\") " pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:28.273766 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271753 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6349a4f6-297b-4718-9981-b1dbc8415f79-cni-binary-copy\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.273766 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271800 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-run-openvswitch\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.273766 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271811 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b439b91-714c-4386-a270-fbb3115347c6-iptables-alerter-script\") pod \"iptables-alerter-f8nv2\" (UID: \"2b439b91-714c-4386-a270-fbb3115347c6\") " pod="openshift-network-operator/iptables-alerter-f8nv2" Jul 08 19:05:28.273766 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271834 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/45b5625b-d7f7-488f-ae0c-c8dd8103c4f9-device-dir\") pod \"aws-ebs-csi-driver-node-bzdv5\" (UID: \"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.273766 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271888 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/45b5625b-d7f7-488f-ae0c-c8dd8103c4f9-device-dir\") pod \"aws-ebs-csi-driver-node-bzdv5\" (UID: \"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.274473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271913 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-etc-sysctl-d\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.274473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271907 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-run-openvswitch\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.274473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271964 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-systemd-units\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.274473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271994 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-run-ovn\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.274473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272023 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-systemd-units\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.274473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272022 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/426767c6-42b0-4249-b95c-d0487aa20faa-ovnkube-script-lib\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.274473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.271997 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fa2c98dd-2141-482f-b02f-9e522908fd71-etc-sysctl-d\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.274473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272067 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4d42\" (UniqueName: \"kubernetes.io/projected/45b5625b-d7f7-488f-ae0c-c8dd8103c4f9-kube-api-access-x4d42\") pod \"aws-ebs-csi-driver-node-bzdv5\" (UID: \"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.274473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272088 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-run-ovn\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.274473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272102 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-host-var-lib-kubelet\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.274473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272163 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-host-run-multus-certs\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.274473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272183 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6349a4f6-297b-4718-9981-b1dbc8415f79-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.274473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272208 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-var-lib-openvswitch\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.274473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272236 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/426767c6-42b0-4249-b95c-d0487aa20faa-ovnkube-config\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.274473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272257 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/426767c6-42b0-4249-b95c-d0487aa20faa-var-lib-openvswitch\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.274473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272292 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/77822693-2416-4400-87cf-4619ac479e79-kubelet-config\") pod \"global-pull-secret-syncer-fmbfk\" (UID: \"77822693-2416-4400-87cf-4619ac479e79\") " pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:28.274473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272310 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/45b5625b-d7f7-488f-ae0c-c8dd8103c4f9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bzdv5\" (UID: \"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.275187 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272341 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/45b5625b-d7f7-488f-ae0c-c8dd8103c4f9-etc-selinux\") pod \"aws-ebs-csi-driver-node-bzdv5\" (UID: \"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.275187 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272369 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-cni-binary-copy\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.275187 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272395 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwx64\" (UniqueName: \"kubernetes.io/projected/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-kube-api-access-rwx64\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.275187 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272409 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/45b5625b-d7f7-488f-ae0c-c8dd8103c4f9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bzdv5\" (UID: \"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.275187 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272423 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76tkd\" (UniqueName: \"kubernetes.io/projected/a7d66e59-7d94-4a00-86f4-fbb8dfe65280-kube-api-access-76tkd\") pod \"node-resolver-sg8ph\" (UID: \"a7d66e59-7d94-4a00-86f4-fbb8dfe65280\") " pod="openshift-dns/node-resolver-sg8ph" Jul 08 19:05:28.275187 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272365 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/77822693-2416-4400-87cf-4619ac479e79-kubelet-config\") pod \"global-pull-secret-syncer-fmbfk\" (UID: \"77822693-2416-4400-87cf-4619ac479e79\") " pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:28.275187 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272472 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/45b5625b-d7f7-488f-ae0c-c8dd8103c4f9-etc-selinux\") pod \"aws-ebs-csi-driver-node-bzdv5\" (UID: \"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.275187 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272515 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fa2c98dd-2141-482f-b02f-9e522908fd71-etc-tuned\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.275187 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272569 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xp7d\" (UniqueName: \"kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d\") pod \"network-check-target-qm8rp\" (UID: \"03cbe6c1-781c-4091-86ca-9de8256eda85\") " pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:28.275187 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272546 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/426767c6-42b0-4249-b95c-d0487aa20faa-ovnkube-script-lib\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.275187 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272622 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-system-cni-dir\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.275187 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272667 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-os-release\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.275187 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272701 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6349a4f6-297b-4718-9981-b1dbc8415f79-system-cni-dir\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.275187 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.272946 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/426767c6-42b0-4249-b95c-d0487aa20faa-ovnkube-config\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.275187 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.273615 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fa2c98dd-2141-482f-b02f-9e522908fd71-tmp\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.275187 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.273693 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/426767c6-42b0-4249-b95c-d0487aa20faa-ovn-node-metrics-cert\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.275187 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.274294 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/69978d76-a5ad-4ebf-9d22-8335ec387429-agent-certs\") pod \"konnectivity-agent-wndp8\" (UID: \"69978d76-a5ad-4ebf-9d22-8335ec387429\") " pod="kube-system/konnectivity-agent-wndp8" Jul 08 19:05:28.275903 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.275791 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fa2c98dd-2141-482f-b02f-9e522908fd71-etc-tuned\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.286034 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.286009 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdvd8\" (UniqueName: \"kubernetes.io/projected/fa2c98dd-2141-482f-b02f-9e522908fd71-kube-api-access-qdvd8\") pod \"tuned-pvrsj\" (UID: \"fa2c98dd-2141-482f-b02f-9e522908fd71\") " pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.286172 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.286065 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zkh7\" (UniqueName: \"kubernetes.io/projected/04ab4794-25e2-4c82-9e44-b460a3993f26-kube-api-access-9zkh7\") pod \"node-ca-xs94k\" (UID: \"04ab4794-25e2-4c82-9e44-b460a3993f26\") " pod="openshift-image-registry/node-ca-xs94k" Jul 08 19:05:28.286172 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.286078 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r97p\" (UniqueName: \"kubernetes.io/projected/2b439b91-714c-4386-a270-fbb3115347c6-kube-api-access-6r97p\") pod \"iptables-alerter-f8nv2\" (UID: \"2b439b91-714c-4386-a270-fbb3115347c6\") " pod="openshift-network-operator/iptables-alerter-f8nv2" Jul 08 19:05:28.286481 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.286455 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4d42\" (UniqueName: \"kubernetes.io/projected/45b5625b-d7f7-488f-ae0c-c8dd8103c4f9-kube-api-access-x4d42\") pod \"aws-ebs-csi-driver-node-bzdv5\" (UID: \"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.288597 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.288572 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bpc5\" (UniqueName: \"kubernetes.io/projected/426767c6-42b0-4249-b95c-d0487aa20faa-kube-api-access-2bpc5\") pod \"ovnkube-node-nklcn\" (UID: \"426767c6-42b0-4249-b95c-d0487aa20faa\") " pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.353429 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.353374 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" event={"ID":"bfc2ad129405fbfb160c1db3475eb64b","Type":"ContainerStarted","Data":"ba3c85ac3f1f69c58f5319cc65a7fb5385424f14764c22f93b4513a5c563a7e6"} Jul 08 19:05:28.354657 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.354630 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal" event={"ID":"301ed78f06a8f545f44b18fb2b42ca58","Type":"ContainerStarted","Data":"784a1db3d746d433d900e6a73a0781221a5da5bef6cd42e5092f5be7d426ab01"} Jul 08 19:05:28.373118 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373085 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76tkd\" (UniqueName: \"kubernetes.io/projected/a7d66e59-7d94-4a00-86f4-fbb8dfe65280-kube-api-access-76tkd\") pod \"node-resolver-sg8ph\" (UID: \"a7d66e59-7d94-4a00-86f4-fbb8dfe65280\") " pod="openshift-dns/node-resolver-sg8ph" Jul 08 19:05:28.373318 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373130 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xp7d\" (UniqueName: \"kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d\") pod \"network-check-target-qm8rp\" (UID: \"03cbe6c1-781c-4091-86ca-9de8256eda85\") " pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:28.373318 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373157 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-system-cni-dir\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.373318 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373176 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-os-release\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.373318 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373194 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6349a4f6-297b-4718-9981-b1dbc8415f79-system-cni-dir\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.373318 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373212 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-multus-cni-dir\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.373318 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373237 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-multus-conf-dir\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.373318 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373257 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs\") pod \"network-metrics-daemon-2bqn6\" (UID: \"3981d8c6-c84c-439a-bf4b-d9834812c8bb\") " pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:28.373318 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373273 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6349a4f6-297b-4718-9981-b1dbc8415f79-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.373318 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373302 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7d66e59-7d94-4a00-86f4-fbb8dfe65280-tmp-dir\") pod \"node-resolver-sg8ph\" (UID: \"a7d66e59-7d94-4a00-86f4-fbb8dfe65280\") " pod="openshift-dns/node-resolver-sg8ph" Jul 08 19:05:28.373820 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373329 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-host-var-lib-cni-bin\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.373820 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373367 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-etc-kubernetes\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.373820 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373386 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6349a4f6-297b-4718-9981-b1dbc8415f79-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.373820 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373402 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nl2d\" (UniqueName: \"kubernetes.io/projected/6349a4f6-297b-4718-9981-b1dbc8415f79-kube-api-access-8nl2d\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.373820 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373422 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-host-run-netns\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.373820 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373437 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-host-var-lib-cni-multus\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.373820 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373454 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-hostroot\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.373820 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373470 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6349a4f6-297b-4718-9981-b1dbc8415f79-os-release\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.373820 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:28.373482 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:05:28.373820 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373494 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-multus-socket-dir-parent\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.373820 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373527 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-multus-daemon-config\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.373820 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373538 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6349a4f6-297b-4718-9981-b1dbc8415f79-system-cni-dir\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.373820 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373553 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a7d66e59-7d94-4a00-86f4-fbb8dfe65280-hosts-file\") pod \"node-resolver-sg8ph\" (UID: \"a7d66e59-7d94-4a00-86f4-fbb8dfe65280\") " pod="openshift-dns/node-resolver-sg8ph" Jul 08 19:05:28.373820 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:28.373582 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs podName:3981d8c6-c84c-439a-bf4b-d9834812c8bb nodeName:}" failed. No retries permitted until 2026-07-08 19:05:28.873559318 +0000 UTC m=+3.051693314 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs") pod "network-metrics-daemon-2bqn6" (UID: "3981d8c6-c84c-439a-bf4b-d9834812c8bb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:05:28.373820 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373645 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-system-cni-dir\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.373820 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373706 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a7d66e59-7d94-4a00-86f4-fbb8dfe65280-hosts-file\") pod \"node-resolver-sg8ph\" (UID: \"a7d66e59-7d94-4a00-86f4-fbb8dfe65280\") " pod="openshift-dns/node-resolver-sg8ph" Jul 08 19:05:28.373820 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373719 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-os-release\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.374734 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373771 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6349a4f6-297b-4718-9981-b1dbc8415f79-cnibin\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.374734 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373811 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-cnibin\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.374734 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373839 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-host-run-k8s-cni-cncf-io\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.374734 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373844 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-multus-cni-dir\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.374734 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373867 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnf7n\" (UniqueName: \"kubernetes.io/projected/3981d8c6-c84c-439a-bf4b-d9834812c8bb-kube-api-access-nnf7n\") pod \"network-metrics-daemon-2bqn6\" (UID: \"3981d8c6-c84c-439a-bf4b-d9834812c8bb\") " pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:28.374734 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373894 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6349a4f6-297b-4718-9981-b1dbc8415f79-cni-binary-copy\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.374734 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373904 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-multus-conf-dir\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.374734 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373924 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-host-var-lib-kubelet\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.374734 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373944 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-host-run-multus-certs\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.374734 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373955 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-host-run-k8s-cni-cncf-io\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.374734 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373962 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6349a4f6-297b-4718-9981-b1dbc8415f79-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.374734 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373994 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-cni-binary-copy\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.374734 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.373997 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6349a4f6-297b-4718-9981-b1dbc8415f79-cnibin\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.374734 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.374020 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwx64\" (UniqueName: \"kubernetes.io/projected/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-kube-api-access-rwx64\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.374734 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.374047 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-cnibin\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.374734 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.374104 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-host-run-multus-certs\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.374734 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.374349 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6349a4f6-297b-4718-9981-b1dbc8415f79-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.375497 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.374396 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6349a4f6-297b-4718-9981-b1dbc8415f79-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.375497 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.374597 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7d66e59-7d94-4a00-86f4-fbb8dfe65280-tmp-dir\") pod \"node-resolver-sg8ph\" (UID: \"a7d66e59-7d94-4a00-86f4-fbb8dfe65280\") " pod="openshift-dns/node-resolver-sg8ph" Jul 08 19:05:28.375497 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.374673 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-host-var-lib-cni-bin\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.375497 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.374674 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6349a4f6-297b-4718-9981-b1dbc8415f79-cni-binary-copy\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.375497 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.374704 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-etc-kubernetes\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.375497 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.374738 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-host-var-lib-kubelet\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.375497 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.374795 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-hostroot\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.375497 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.374978 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-cni-binary-copy\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.375497 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.375046 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-host-var-lib-cni-multus\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.375497 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.375079 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6349a4f6-297b-4718-9981-b1dbc8415f79-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.375497 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.375115 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-multus-socket-dir-parent\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.375497 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.375184 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6349a4f6-297b-4718-9981-b1dbc8415f79-os-release\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.375497 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.375222 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-host-run-netns\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.376015 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.375578 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-multus-daemon-config\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.389265 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:28.389189 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:05:28.389265 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:28.389218 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:05:28.389265 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:28.389231 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9xp7d for pod openshift-network-diagnostics/network-check-target-qm8rp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:05:28.389516 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:28.389301 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d podName:03cbe6c1-781c-4091-86ca-9de8256eda85 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:28.889280547 +0000 UTC m=+3.067414547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9xp7d" (UniqueName: "kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d") pod "network-check-target-qm8rp" (UID: "03cbe6c1-781c-4091-86ca-9de8256eda85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:05:28.392165 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.392129 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76tkd\" (UniqueName: \"kubernetes.io/projected/a7d66e59-7d94-4a00-86f4-fbb8dfe65280-kube-api-access-76tkd\") pod \"node-resolver-sg8ph\" (UID: \"a7d66e59-7d94-4a00-86f4-fbb8dfe65280\") " pod="openshift-dns/node-resolver-sg8ph" Jul 08 19:05:28.398441 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.398415 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnf7n\" (UniqueName: \"kubernetes.io/projected/3981d8c6-c84c-439a-bf4b-d9834812c8bb-kube-api-access-nnf7n\") pod \"network-metrics-daemon-2bqn6\" (UID: \"3981d8c6-c84c-439a-bf4b-d9834812c8bb\") " pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:28.398441 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.398428 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwx64\" (UniqueName: \"kubernetes.io/projected/c80f1ff4-3d7b-49dc-9320-e6ecba209dc1-kube-api-access-rwx64\") pod \"multus-5sxcv\" (UID: \"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1\") " pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.401596 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.401568 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nl2d\" (UniqueName: \"kubernetes.io/projected/6349a4f6-297b-4718-9981-b1dbc8415f79-kube-api-access-8nl2d\") pod \"multus-additional-cni-plugins-5wxhc\" (UID: \"6349a4f6-297b-4718-9981-b1dbc8415f79\") " pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.461953 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.461912 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" Jul 08 19:05:28.475046 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.475013 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wndp8" Jul 08 19:05:28.484964 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.484930 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f8nv2" Jul 08 19:05:28.488970 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.488952 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:28.496320 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.496298 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" Jul 08 19:05:28.503896 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.503872 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xs94k" Jul 08 19:05:28.511286 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.511262 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-sg8ph" Jul 08 19:05:28.518183 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.518161 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5sxcv" Jul 08 19:05:28.523803 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.523765 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5wxhc" Jul 08 19:05:28.698701 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.698624 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 19:05:28.778253 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.778209 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret\") pod \"global-pull-secret-syncer-fmbfk\" (UID: \"77822693-2416-4400-87cf-4619ac479e79\") " pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:28.778446 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:28.778432 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:05:28.778500 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:28.778494 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret podName:77822693-2416-4400-87cf-4619ac479e79 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:29.778479273 +0000 UTC m=+3.956613260 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret") pod "global-pull-secret-syncer-fmbfk" (UID: "77822693-2416-4400-87cf-4619ac479e79") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:05:28.879214 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.879177 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs\") pod \"network-metrics-daemon-2bqn6\" (UID: \"3981d8c6-c84c-439a-bf4b-d9834812c8bb\") " pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:28.879388 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:28.879327 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:05:28.879467 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:28.879395 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs podName:3981d8c6-c84c-439a-bf4b-d9834812c8bb nodeName:}" failed. No retries permitted until 2026-07-08 19:05:29.879375532 +0000 UTC m=+4.057509528 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs") pod "network-metrics-daemon-2bqn6" (UID: "3981d8c6-c84c-439a-bf4b-d9834812c8bb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:05:28.979851 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:28.979770 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xp7d\" (UniqueName: \"kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d\") pod \"network-check-target-qm8rp\" (UID: \"03cbe6c1-781c-4091-86ca-9de8256eda85\") " pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:28.980022 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:28.979910 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:05:28.980022 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:28.979935 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:05:28.980022 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:28.979950 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9xp7d for pod openshift-network-diagnostics/network-check-target-qm8rp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:05:28.980022 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:28.980015 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d podName:03cbe6c1-781c-4091-86ca-9de8256eda85 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:29.979994389 +0000 UTC m=+4.158128399 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9xp7d" (UniqueName: "kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d") pod "network-check-target-qm8rp" (UID: "03cbe6c1-781c-4091-86ca-9de8256eda85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:05:29.205653 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:29.205608 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 19:00:28 +0000 UTC" deadline="2027-12-03 18:23:11.228973826 +0000 UTC" Jul 08 19:05:29.205653 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:29.205648 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12311h17m42.023329256s" Jul 08 19:05:29.348891 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:29.348854 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:29.349055 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:29.348979 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fmbfk" podUID="77822693-2416-4400-87cf-4619ac479e79" Jul 08 19:05:29.560382 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:29.560325 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69978d76_a5ad_4ebf_9d22_8335ec387429.slice/crio-57118200e0f4a88068f63fc88ba6c26183b73dbc28cfd8bce8e545d2bbfcc35b WatchSource:0}: Error finding container 57118200e0f4a88068f63fc88ba6c26183b73dbc28cfd8bce8e545d2bbfcc35b: Status 404 returned error can't find the container with id 57118200e0f4a88068f63fc88ba6c26183b73dbc28cfd8bce8e545d2bbfcc35b Jul 08 19:05:29.562844 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:29.562526 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45b5625b_d7f7_488f_ae0c_c8dd8103c4f9.slice/crio-fd2506242c2af3c417b930868f3b2cbb1d297b0cc9c5fc6f80f73b71c73f777b WatchSource:0}: Error finding container fd2506242c2af3c417b930868f3b2cbb1d297b0cc9c5fc6f80f73b71c73f777b: Status 404 returned error can't find the container with id fd2506242c2af3c417b930868f3b2cbb1d297b0cc9c5fc6f80f73b71c73f777b Jul 08 19:05:29.563173 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:29.563076 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa2c98dd_2141_482f_b02f_9e522908fd71.slice/crio-1f63dabb331a4a5d47fbd0a8458c7c44c65ced3c35488555dedb07ea250e64f8 WatchSource:0}: Error finding container 1f63dabb331a4a5d47fbd0a8458c7c44c65ced3c35488555dedb07ea250e64f8: Status 404 returned error can't find the container with id 1f63dabb331a4a5d47fbd0a8458c7c44c65ced3c35488555dedb07ea250e64f8 Jul 08 19:05:29.568238 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:29.568097 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6349a4f6_297b_4718_9981_b1dbc8415f79.slice/crio-f77c973204ea2bfbb22390fafe1f4f0c0eddd7804f771de4fb0cf4880dc8c431 WatchSource:0}: Error finding container f77c973204ea2bfbb22390fafe1f4f0c0eddd7804f771de4fb0cf4880dc8c431: Status 404 returned error can't find the container with id f77c973204ea2bfbb22390fafe1f4f0c0eddd7804f771de4fb0cf4880dc8c431 Jul 08 19:05:29.575443 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:29.575413 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04ab4794_25e2_4c82_9e44_b460a3993f26.slice/crio-5d7a2debcd8f7ac99c5ff1f9a41ede5ffebcc86ef3c65fc4e42eb443df9ad6c0 WatchSource:0}: Error finding container 5d7a2debcd8f7ac99c5ff1f9a41ede5ffebcc86ef3c65fc4e42eb443df9ad6c0: Status 404 returned error can't find the container with id 5d7a2debcd8f7ac99c5ff1f9a41ede5ffebcc86ef3c65fc4e42eb443df9ad6c0 Jul 08 19:05:29.575684 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:29.575645 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7d66e59_7d94_4a00_86f4_fbb8dfe65280.slice/crio-44772b170f2a54b31590818f914a5c2022951c7ba54b0c0f645e5ffb4c68083e WatchSource:0}: Error finding container 44772b170f2a54b31590818f914a5c2022951c7ba54b0c0f645e5ffb4c68083e: Status 404 returned error can't find the container with id 44772b170f2a54b31590818f914a5c2022951c7ba54b0c0f645e5ffb4c68083e Jul 08 19:05:29.577487 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:29.577444 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b439b91_714c_4386_a270_fbb3115347c6.slice/crio-2ca8345d8c27005e32d48866eb933ffdfe4405a0518bbc73cbbb0bf425c61464 WatchSource:0}: Error finding container 2ca8345d8c27005e32d48866eb933ffdfe4405a0518bbc73cbbb0bf425c61464: Status 404 returned error can't find the container with id 2ca8345d8c27005e32d48866eb933ffdfe4405a0518bbc73cbbb0bf425c61464 Jul 08 19:05:29.579400 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:29.579310 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc80f1ff4_3d7b_49dc_9320_e6ecba209dc1.slice/crio-208e4a266bd98b22e951097495ea55ce5e3bc7cc5ee117a5dfc0ffae4adbcd34 WatchSource:0}: Error finding container 208e4a266bd98b22e951097495ea55ce5e3bc7cc5ee117a5dfc0ffae4adbcd34: Status 404 returned error can't find the container with id 208e4a266bd98b22e951097495ea55ce5e3bc7cc5ee117a5dfc0ffae4adbcd34 Jul 08 19:05:29.580360 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:05:29.580342 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod426767c6_42b0_4249_b95c_d0487aa20faa.slice/crio-3b47220503853e815f432cc56af9f9fdef02c309491606b2b2c85b2f1dcdfdf7 WatchSource:0}: Error finding container 3b47220503853e815f432cc56af9f9fdef02c309491606b2b2c85b2f1dcdfdf7: Status 404 returned error can't find the container with id 3b47220503853e815f432cc56af9f9fdef02c309491606b2b2c85b2f1dcdfdf7 Jul 08 19:05:29.785272 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:29.785229 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret\") pod \"global-pull-secret-syncer-fmbfk\" (UID: \"77822693-2416-4400-87cf-4619ac479e79\") " pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:29.785404 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:29.785347 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:05:29.785439 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:29.785405 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret podName:77822693-2416-4400-87cf-4619ac479e79 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:31.785390129 +0000 UTC m=+5.963524115 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret") pod "global-pull-secret-syncer-fmbfk" (UID: "77822693-2416-4400-87cf-4619ac479e79") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:05:29.886003 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:29.885972 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs\") pod \"network-metrics-daemon-2bqn6\" (UID: \"3981d8c6-c84c-439a-bf4b-d9834812c8bb\") " pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:29.886156 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:29.886129 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:05:29.886207 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:29.886195 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs podName:3981d8c6-c84c-439a-bf4b-d9834812c8bb nodeName:}" failed. No retries permitted until 2026-07-08 19:05:31.886179637 +0000 UTC m=+6.064313622 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs") pod "network-metrics-daemon-2bqn6" (UID: "3981d8c6-c84c-439a-bf4b-d9834812c8bb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:05:29.987277 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:29.987233 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xp7d\" (UniqueName: \"kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d\") pod \"network-check-target-qm8rp\" (UID: \"03cbe6c1-781c-4091-86ca-9de8256eda85\") " pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:29.987418 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:29.987395 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:05:29.987418 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:29.987418 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:05:29.987509 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:29.987431 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9xp7d for pod openshift-network-diagnostics/network-check-target-qm8rp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:05:29.987509 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:29.987482 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d podName:03cbe6c1-781c-4091-86ca-9de8256eda85 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:31.987468353 +0000 UTC m=+6.165602341 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9xp7d" (UniqueName: "kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d") pod "network-check-target-qm8rp" (UID: "03cbe6c1-781c-4091-86ca-9de8256eda85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:05:30.206285 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:30.206141 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 19:00:28 +0000 UTC" deadline="2028-03-20 23:49:48.16669045 +0000 UTC" Jul 08 19:05:30.206285 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:30.206184 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14908h44m17.960510775s" Jul 08 19:05:30.352635 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:30.352060 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:30.352635 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:30.352164 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qm8rp" podUID="03cbe6c1-781c-4091-86ca-9de8256eda85" Jul 08 19:05:30.352635 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:30.352472 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:30.353043 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:30.352990 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bqn6" podUID="3981d8c6-c84c-439a-bf4b-d9834812c8bb" Jul 08 19:05:30.379114 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:30.379071 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal" event={"ID":"301ed78f06a8f545f44b18fb2b42ca58","Type":"ContainerStarted","Data":"fda10f5e1fea020d86563840acaf545c05422dc8fd8aea670ebc1f4508c77f23"} Jul 08 19:05:30.381334 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:30.381199 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" event={"ID":"426767c6-42b0-4249-b95c-d0487aa20faa","Type":"ContainerStarted","Data":"3b47220503853e815f432cc56af9f9fdef02c309491606b2b2c85b2f1dcdfdf7"} Jul 08 19:05:30.391018 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:30.390879 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5sxcv" event={"ID":"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1","Type":"ContainerStarted","Data":"208e4a266bd98b22e951097495ea55ce5e3bc7cc5ee117a5dfc0ffae4adbcd34"} Jul 08 19:05:30.393158 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:30.393082 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f8nv2" event={"ID":"2b439b91-714c-4386-a270-fbb3115347c6","Type":"ContainerStarted","Data":"2ca8345d8c27005e32d48866eb933ffdfe4405a0518bbc73cbbb0bf425c61464"} Jul 08 19:05:30.408374 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:30.408280 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sg8ph" event={"ID":"a7d66e59-7d94-4a00-86f4-fbb8dfe65280","Type":"ContainerStarted","Data":"44772b170f2a54b31590818f914a5c2022951c7ba54b0c0f645e5ffb4c68083e"} Jul 08 19:05:30.417944 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:30.417887 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xs94k" event={"ID":"04ab4794-25e2-4c82-9e44-b460a3993f26","Type":"ContainerStarted","Data":"5d7a2debcd8f7ac99c5ff1f9a41ede5ffebcc86ef3c65fc4e42eb443df9ad6c0"} Jul 08 19:05:30.420250 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:30.420222 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" event={"ID":"fa2c98dd-2141-482f-b02f-9e522908fd71","Type":"ContainerStarted","Data":"1f63dabb331a4a5d47fbd0a8458c7c44c65ced3c35488555dedb07ea250e64f8"} Jul 08 19:05:30.423883 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:30.423858 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" event={"ID":"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9","Type":"ContainerStarted","Data":"fd2506242c2af3c417b930868f3b2cbb1d297b0cc9c5fc6f80f73b71c73f777b"} Jul 08 19:05:30.430072 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:30.430032 2560 generic.go:356] "Generic (PLEG): container finished" podID="bfc2ad129405fbfb160c1db3475eb64b" containerID="61504c0686f6d905c36c1893a195169ba0ab2abf6bbdc760ad6524e679283baa" exitCode=0 Jul 08 19:05:30.430182 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:30.430124 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" event={"ID":"bfc2ad129405fbfb160c1db3475eb64b","Type":"ContainerDied","Data":"61504c0686f6d905c36c1893a195169ba0ab2abf6bbdc760ad6524e679283baa"} Jul 08 19:05:30.437014 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:30.436951 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wxhc" event={"ID":"6349a4f6-297b-4718-9981-b1dbc8415f79","Type":"ContainerStarted","Data":"f77c973204ea2bfbb22390fafe1f4f0c0eddd7804f771de4fb0cf4880dc8c431"} Jul 08 19:05:30.442770 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:30.442726 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wndp8" event={"ID":"69978d76-a5ad-4ebf-9d22-8335ec387429","Type":"ContainerStarted","Data":"57118200e0f4a88068f63fc88ba6c26183b73dbc28cfd8bce8e545d2bbfcc35b"} Jul 08 19:05:30.448349 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:30.448133 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-38.ec2.internal" podStartSLOduration=3.4481193980000002 podStartE2EDuration="3.448119398s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:30.398381828 +0000 UTC m=+4.576515836" watchObservedRunningTime="2026-07-08 19:05:30.448119398 +0000 UTC m=+4.626253403" Jul 08 19:05:31.349195 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:31.349152 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:31.349765 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:31.349289 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fmbfk" podUID="77822693-2416-4400-87cf-4619ac479e79" Jul 08 19:05:31.452067 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:31.452019 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" event={"ID":"bfc2ad129405fbfb160c1db3475eb64b","Type":"ContainerStarted","Data":"0fa9235e034f9d8d6d9bd3ea75f10695b921720745b1d4dad95b60d9aacfc9c0"} Jul 08 19:05:31.800882 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:31.800787 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret\") pod \"global-pull-secret-syncer-fmbfk\" (UID: \"77822693-2416-4400-87cf-4619ac479e79\") " pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:31.801047 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:31.800960 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:05:31.801047 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:31.801036 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret podName:77822693-2416-4400-87cf-4619ac479e79 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:35.801006175 +0000 UTC m=+9.979140163 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret") pod "global-pull-secret-syncer-fmbfk" (UID: "77822693-2416-4400-87cf-4619ac479e79") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:05:31.901262 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:31.901215 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs\") pod \"network-metrics-daemon-2bqn6\" (UID: \"3981d8c6-c84c-439a-bf4b-d9834812c8bb\") " pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:31.901444 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:31.901378 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:05:31.901506 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:31.901453 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs podName:3981d8c6-c84c-439a-bf4b-d9834812c8bb nodeName:}" failed. No retries permitted until 2026-07-08 19:05:35.901432374 +0000 UTC m=+10.079566363 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs") pod "network-metrics-daemon-2bqn6" (UID: "3981d8c6-c84c-439a-bf4b-d9834812c8bb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:05:32.002547 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:32.002494 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xp7d\" (UniqueName: \"kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d\") pod \"network-check-target-qm8rp\" (UID: \"03cbe6c1-781c-4091-86ca-9de8256eda85\") " pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:32.002743 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:32.002726 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:05:32.002820 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:32.002751 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:05:32.002820 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:32.002764 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9xp7d for pod openshift-network-diagnostics/network-check-target-qm8rp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:05:32.002928 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:32.002823 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d podName:03cbe6c1-781c-4091-86ca-9de8256eda85 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:36.00280507 +0000 UTC m=+10.180939062 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9xp7d" (UniqueName: "kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d") pod "network-check-target-qm8rp" (UID: "03cbe6c1-781c-4091-86ca-9de8256eda85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:05:32.350998 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:32.350969 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:32.351451 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:32.351064 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qm8rp" podUID="03cbe6c1-781c-4091-86ca-9de8256eda85" Jul 08 19:05:32.351451 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:32.351378 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:32.351451 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:32.351448 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bqn6" podUID="3981d8c6-c84c-439a-bf4b-d9834812c8bb" Jul 08 19:05:33.348885 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:33.348853 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:33.349071 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:33.348980 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fmbfk" podUID="77822693-2416-4400-87cf-4619ac479e79" Jul 08 19:05:34.349092 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:34.349056 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:34.349562 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:34.349189 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qm8rp" podUID="03cbe6c1-781c-4091-86ca-9de8256eda85" Jul 08 19:05:34.349852 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:34.349831 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:34.349999 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:34.349981 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bqn6" podUID="3981d8c6-c84c-439a-bf4b-d9834812c8bb" Jul 08 19:05:35.348748 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:35.348713 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:35.348928 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:35.348848 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fmbfk" podUID="77822693-2416-4400-87cf-4619ac479e79" Jul 08 19:05:35.836215 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:35.836117 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret\") pod \"global-pull-secret-syncer-fmbfk\" (UID: \"77822693-2416-4400-87cf-4619ac479e79\") " pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:35.836712 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:35.836294 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:05:35.836712 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:35.836382 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret podName:77822693-2416-4400-87cf-4619ac479e79 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:43.836359871 +0000 UTC m=+18.014493874 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret") pod "global-pull-secret-syncer-fmbfk" (UID: "77822693-2416-4400-87cf-4619ac479e79") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:05:35.936602 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:35.936514 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs\") pod \"network-metrics-daemon-2bqn6\" (UID: \"3981d8c6-c84c-439a-bf4b-d9834812c8bb\") " pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:35.936782 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:35.936699 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:05:35.936782 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:35.936779 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs podName:3981d8c6-c84c-439a-bf4b-d9834812c8bb nodeName:}" failed. No retries permitted until 2026-07-08 19:05:43.936758974 +0000 UTC m=+18.114892978 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs") pod "network-metrics-daemon-2bqn6" (UID: "3981d8c6-c84c-439a-bf4b-d9834812c8bb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:05:36.037606 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:36.037458 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xp7d\" (UniqueName: \"kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d\") pod \"network-check-target-qm8rp\" (UID: \"03cbe6c1-781c-4091-86ca-9de8256eda85\") " pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:36.037769 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:36.037654 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:05:36.037769 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:36.037682 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:05:36.037769 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:36.037697 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9xp7d for pod openshift-network-diagnostics/network-check-target-qm8rp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:05:36.037769 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:36.037761 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d podName:03cbe6c1-781c-4091-86ca-9de8256eda85 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:44.037741855 +0000 UTC m=+18.215875861 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9xp7d" (UniqueName: "kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d") pod "network-check-target-qm8rp" (UID: "03cbe6c1-781c-4091-86ca-9de8256eda85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:05:36.350973 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:36.350391 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:36.350973 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:36.350510 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qm8rp" podUID="03cbe6c1-781c-4091-86ca-9de8256eda85" Jul 08 19:05:36.350973 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:36.350568 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:36.350973 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:36.350649 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bqn6" podUID="3981d8c6-c84c-439a-bf4b-d9834812c8bb" Jul 08 19:05:37.348724 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:37.348692 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:37.349154 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:37.348820 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fmbfk" podUID="77822693-2416-4400-87cf-4619ac479e79" Jul 08 19:05:38.349599 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:38.349566 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:38.350039 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:38.349579 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:38.350039 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:38.349704 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qm8rp" podUID="03cbe6c1-781c-4091-86ca-9de8256eda85" Jul 08 19:05:38.350039 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:38.349738 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bqn6" podUID="3981d8c6-c84c-439a-bf4b-d9834812c8bb" Jul 08 19:05:39.349707 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:39.349210 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:39.349707 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:39.349346 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fmbfk" podUID="77822693-2416-4400-87cf-4619ac479e79" Jul 08 19:05:40.351142 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:40.351113 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:40.351546 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:40.351226 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qm8rp" podUID="03cbe6c1-781c-4091-86ca-9de8256eda85" Jul 08 19:05:40.351621 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:40.351582 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:40.351694 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:40.351672 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bqn6" podUID="3981d8c6-c84c-439a-bf4b-d9834812c8bb" Jul 08 19:05:41.348849 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:41.348818 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:41.348993 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:41.348938 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fmbfk" podUID="77822693-2416-4400-87cf-4619ac479e79" Jul 08 19:05:42.349218 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:42.349181 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:42.349707 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:42.349235 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:42.349707 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:42.349347 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bqn6" podUID="3981d8c6-c84c-439a-bf4b-d9834812c8bb" Jul 08 19:05:42.349707 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:42.349464 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qm8rp" podUID="03cbe6c1-781c-4091-86ca-9de8256eda85" Jul 08 19:05:43.349327 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:43.349291 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:43.349817 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:43.349431 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fmbfk" podUID="77822693-2416-4400-87cf-4619ac479e79" Jul 08 19:05:43.897715 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:43.897677 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret\") pod \"global-pull-secret-syncer-fmbfk\" (UID: \"77822693-2416-4400-87cf-4619ac479e79\") " pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:43.898690 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:43.898664 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:05:43.898796 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:43.898752 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret podName:77822693-2416-4400-87cf-4619ac479e79 nodeName:}" failed. No retries permitted until 2026-07-08 19:05:59.898731657 +0000 UTC m=+34.076865660 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret") pod "global-pull-secret-syncer-fmbfk" (UID: "77822693-2416-4400-87cf-4619ac479e79") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:05:43.998315 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:43.998263 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs\") pod \"network-metrics-daemon-2bqn6\" (UID: \"3981d8c6-c84c-439a-bf4b-d9834812c8bb\") " pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:43.998508 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:43.998429 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:05:43.998508 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:43.998497 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs podName:3981d8c6-c84c-439a-bf4b-d9834812c8bb nodeName:}" failed. No retries permitted until 2026-07-08 19:05:59.998477662 +0000 UTC m=+34.176611648 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs") pod "network-metrics-daemon-2bqn6" (UID: "3981d8c6-c84c-439a-bf4b-d9834812c8bb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:05:44.099358 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:44.099317 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xp7d\" (UniqueName: \"kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d\") pod \"network-check-target-qm8rp\" (UID: \"03cbe6c1-781c-4091-86ca-9de8256eda85\") " pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:44.099514 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:44.099481 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:05:44.099514 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:44.099509 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:05:44.099640 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:44.099538 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9xp7d for pod openshift-network-diagnostics/network-check-target-qm8rp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:05:44.099640 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:44.099609 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d podName:03cbe6c1-781c-4091-86ca-9de8256eda85 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:00.099589699 +0000 UTC m=+34.277723703 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9xp7d" (UniqueName: "kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d") pod "network-check-target-qm8rp" (UID: "03cbe6c1-781c-4091-86ca-9de8256eda85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:05:44.348636 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:44.348599 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:44.348818 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:44.348723 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qm8rp" podUID="03cbe6c1-781c-4091-86ca-9de8256eda85" Jul 08 19:05:44.349176 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:44.349150 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:44.349320 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:44.349292 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bqn6" podUID="3981d8c6-c84c-439a-bf4b-d9834812c8bb" Jul 08 19:05:45.349214 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:45.349175 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:45.349668 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:45.349307 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fmbfk" podUID="77822693-2416-4400-87cf-4619ac479e79" Jul 08 19:05:46.350544 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:46.350492 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:46.350933 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:46.350647 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bqn6" podUID="3981d8c6-c84c-439a-bf4b-d9834812c8bb" Jul 08 19:05:46.350933 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:46.350700 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:46.350933 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:46.350796 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qm8rp" podUID="03cbe6c1-781c-4091-86ca-9de8256eda85" Jul 08 19:05:47.349557 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:47.349512 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:47.349736 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:47.349649 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fmbfk" podUID="77822693-2416-4400-87cf-4619ac479e79" Jul 08 19:05:48.348753 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:48.348719 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:48.349288 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:48.348863 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bqn6" podUID="3981d8c6-c84c-439a-bf4b-d9834812c8bb" Jul 08 19:05:48.349288 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:48.348923 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:48.349288 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:48.349031 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qm8rp" podUID="03cbe6c1-781c-4091-86ca-9de8256eda85" Jul 08 19:05:49.349931 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:49.349607 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:49.350711 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:49.350018 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fmbfk" podUID="77822693-2416-4400-87cf-4619ac479e79" Jul 08 19:05:49.491138 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:49.491107 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" event={"ID":"426767c6-42b0-4249-b95c-d0487aa20faa","Type":"ContainerStarted","Data":"1a0efbf3afa595c317dcc446100baf9bd6116b031103ad4718bfb2cf868bc3e1"} Jul 08 19:05:49.491354 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:49.491198 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" event={"ID":"426767c6-42b0-4249-b95c-d0487aa20faa","Type":"ContainerStarted","Data":"37023b8ab4555cbc7a81caecebe15ce36667a48d2151dc3fbe8017507440213d"} Jul 08 19:05:49.491354 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:49.491248 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" event={"ID":"426767c6-42b0-4249-b95c-d0487aa20faa","Type":"ContainerStarted","Data":"e3cf94fa2815cc551134676950b773e1131d90d807c5d750c49268e2c47ce7a7"} Jul 08 19:05:49.491354 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:49.491301 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" event={"ID":"426767c6-42b0-4249-b95c-d0487aa20faa","Type":"ContainerStarted","Data":"075dbf4610c7c6927f354763402ff4a6a8c0e833117b4977c2bcd8334238188d"} Jul 08 19:05:49.491354 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:49.491336 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" event={"ID":"426767c6-42b0-4249-b95c-d0487aa20faa","Type":"ContainerStarted","Data":"e0267773eef3bb2694ef91488a1b3070b4fbaa1dca89f72771b69580fe5fcb5f"} Jul 08 19:05:49.491669 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:49.491378 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" event={"ID":"426767c6-42b0-4249-b95c-d0487aa20faa","Type":"ContainerStarted","Data":"d3064ba9aca8da7e0f9a4ff689c0a9445816b1b1dae80e584fe4fbf316e534dc"} Jul 08 19:05:49.492939 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:49.492908 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5sxcv" event={"ID":"c80f1ff4-3d7b-49dc-9320-e6ecba209dc1","Type":"ContainerStarted","Data":"0d39452c22670360043a28a7095780f053fecc1448745c858a1b24e8fbc89a10"} Jul 08 19:05:49.494484 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:49.494459 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-sg8ph" event={"ID":"a7d66e59-7d94-4a00-86f4-fbb8dfe65280","Type":"ContainerStarted","Data":"02b6e33d5ac2e703070dd28311b5bd6cdc4ffe5dcda418b475befeac051a98ac"} Jul 08 19:05:49.495964 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:49.495936 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xs94k" event={"ID":"04ab4794-25e2-4c82-9e44-b460a3993f26","Type":"ContainerStarted","Data":"42bffb13f701d844edbd80593ae731eabbbe3f92e0035d3117ea2bcc3d693ffb"} Jul 08 19:05:49.498136 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:49.498114 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" event={"ID":"fa2c98dd-2141-482f-b02f-9e522908fd71","Type":"ContainerStarted","Data":"52519cc47ebabe453c1bf645fecaa6481e603034984c91b085efcdea4619121c"} Jul 08 19:05:49.499679 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:49.499657 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" event={"ID":"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9","Type":"ContainerStarted","Data":"3a03c753019fecb75bf0c86fd21542adc929f38d23910b506c313d22d65010ba"} Jul 08 19:05:49.501042 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:49.501019 2560 generic.go:356] "Generic (PLEG): container finished" podID="6349a4f6-297b-4718-9981-b1dbc8415f79" containerID="ea7c66b3879bb4208f7116ba13bbe5aad5db7eda112ac08633da8867c7635bcd" exitCode=0 Jul 08 19:05:49.501146 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:49.501092 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wxhc" event={"ID":"6349a4f6-297b-4718-9981-b1dbc8415f79","Type":"ContainerDied","Data":"ea7c66b3879bb4208f7116ba13bbe5aad5db7eda112ac08633da8867c7635bcd"} Jul 08 19:05:49.502634 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:49.502601 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wndp8" event={"ID":"69978d76-a5ad-4ebf-9d22-8335ec387429","Type":"ContainerStarted","Data":"ab67665fb48531a953c87225b9bdf087b3eee854a21907ea1adf4f1a437ce1a2"} Jul 08 19:05:49.513875 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:49.513829 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5sxcv" podStartSLOduration=4.615074626 podStartE2EDuration="23.513799973s" totalImagesPullingTime="18.898725347s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:49.513725547 +0000 UTC m=+23.691859555" watchObservedRunningTime="2026-07-08 19:05:49.513799973 +0000 UTC m=+23.691933970" Jul 08 19:05:49.514197 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:49.514167 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-38.ec2.internal" podStartSLOduration=22.514157159 podStartE2EDuration="22.514157159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:31.47292635 +0000 UTC m=+5.651060357" watchObservedRunningTime="2026-07-08 19:05:49.514157159 +0000 UTC m=+23.692291165" Jul 08 19:05:49.566890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:49.566829 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xs94k" podStartSLOduration=4.706986443 podStartE2EDuration="23.566811089s" totalImagesPullingTime="18.859824646s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:49.536886063 +0000 UTC m=+23.715020070" watchObservedRunningTime="2026-07-08 19:05:49.566811089 +0000 UTC m=+23.744945099" Jul 08 19:05:49.567217 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:49.567180 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wndp8" podStartSLOduration=12.829981214 podStartE2EDuration="23.567170962s" totalImagesPullingTime="10.737189748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:49.567060663 +0000 UTC m=+23.745194669" watchObservedRunningTime="2026-07-08 19:05:49.567170962 +0000 UTC m=+23.745304968" Jul 08 19:05:49.608550 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:49.608473 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-sg8ph" podStartSLOduration=4.762377067 podStartE2EDuration="23.608450245s" totalImagesPullingTime="18.846073178s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:49.585067243 +0000 UTC m=+23.763201249" watchObservedRunningTime="2026-07-08 19:05:49.608450245 +0000 UTC m=+23.786584258" Jul 08 19:05:49.636241 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:49.636187 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pvrsj" podStartSLOduration=4.763871368 podStartE2EDuration="23.636170896s" totalImagesPullingTime="18.872299528s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:49.635879896 +0000 UTC m=+23.814013902" watchObservedRunningTime="2026-07-08 19:05:49.636170896 +0000 UTC m=+23.814304903" Jul 08 19:05:50.123898 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:50.123870 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 19:05:50.231728 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:50.231586 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T19:05:50.123891399Z","UUID":"03abfa86-47e9-459f-b62d-85a07f9d2117","Handler":null,"Name":"","Endpoint":""} Jul 08 19:05:50.233015 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:50.232996 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 19:05:50.233112 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:50.233032 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 19:05:50.348958 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:50.348887 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:50.348958 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:50.348930 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:50.349282 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:50.349017 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qm8rp" podUID="03cbe6c1-781c-4091-86ca-9de8256eda85" Jul 08 19:05:50.349282 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:50.349177 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bqn6" podUID="3981d8c6-c84c-439a-bf4b-d9834812c8bb" Jul 08 19:05:50.507165 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:50.507085 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" event={"ID":"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9","Type":"ContainerStarted","Data":"ffdcf418d2fb1167adf0c7b16c67f3dcb4b1e22810c296636efc098128aca3a4"} Jul 08 19:05:50.508457 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:50.508426 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f8nv2" event={"ID":"2b439b91-714c-4386-a270-fbb3115347c6","Type":"ContainerStarted","Data":"496e1c103883dea97af755dcc00f2ba3592ffd1af6d9402ccbfaebd15ab92280"} Jul 08 19:05:51.349440 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:51.349371 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:51.349651 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:51.349480 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fmbfk" podUID="77822693-2416-4400-87cf-4619ac479e79" Jul 08 19:05:51.514118 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:51.514084 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" event={"ID":"426767c6-42b0-4249-b95c-d0487aa20faa","Type":"ContainerStarted","Data":"e0255a9e963a8afd04d5ebe07592323c7ad604eadc09027f3a1fa2cd96aa4b23"} Jul 08 19:05:52.326741 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:52.326583 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wndp8" Jul 08 19:05:52.327160 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:52.327136 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wndp8" Jul 08 19:05:52.346238 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:52.346198 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-f8nv2" podStartSLOduration=11.733747158 podStartE2EDuration="26.346184436s" totalImagesPullingTime="14.612437278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:50.5314102 +0000 UTC m=+24.709544209" watchObservedRunningTime="2026-07-08 19:05:52.346184436 +0000 UTC m=+26.524318441" Jul 08 19:05:52.349204 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:52.349184 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:52.349316 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:52.349294 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qm8rp" podUID="03cbe6c1-781c-4091-86ca-9de8256eda85" Jul 08 19:05:52.349399 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:52.349383 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:52.349510 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:52.349492 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bqn6" podUID="3981d8c6-c84c-439a-bf4b-d9834812c8bb" Jul 08 19:05:52.517864 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:52.517809 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" event={"ID":"45b5625b-d7f7-488f-ae0c-c8dd8103c4f9","Type":"ContainerStarted","Data":"c27e4d125fe464cfe1c3ff93f1b47cb3d80e8d201b87db5e13a60f5884954c7f"} Jul 08 19:05:52.518283 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:52.518008 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wndp8" Jul 08 19:05:52.518519 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:52.518496 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wndp8" Jul 08 19:05:52.572722 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:52.572675 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bzdv5" podStartSLOduration=5.00785251 podStartE2EDuration="26.572659005s" totalImagesPullingTime="21.564806495s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:52.540229762 +0000 UTC m=+26.718363768" watchObservedRunningTime="2026-07-08 19:05:52.572659005 +0000 UTC m=+26.750793011" Jul 08 19:05:53.349112 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:53.349089 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:53.349237 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:53.349185 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fmbfk" podUID="77822693-2416-4400-87cf-4619ac479e79" Jul 08 19:05:53.524284 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:53.524168 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" event={"ID":"426767c6-42b0-4249-b95c-d0487aa20faa","Type":"ContainerStarted","Data":"93b5dbaa546581808096c565d6ed43ded38943d52b8e7d5d190111a0ff49ecc5"} Jul 08 19:05:53.553476 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:53.553427 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" podStartSLOduration=8.632450985 podStartE2EDuration="27.553410719s" totalImagesPullingTime="18.920959734s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:05:53.552774309 +0000 UTC m=+27.730908320" watchObservedRunningTime="2026-07-08 19:05:53.553410719 +0000 UTC m=+27.731544725" Jul 08 19:05:54.349431 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:54.349346 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:54.349600 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:54.349346 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:54.349600 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:54.349491 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bqn6" podUID="3981d8c6-c84c-439a-bf4b-d9834812c8bb" Jul 08 19:05:54.349600 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:54.349522 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qm8rp" podUID="03cbe6c1-781c-4091-86ca-9de8256eda85" Jul 08 19:05:54.526869 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:54.526838 2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 08 19:05:54.527249 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:54.527177 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:54.527249 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:54.527243 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:54.545454 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:54.545224 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:54.546506 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:54.546482 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:55.348882 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:55.348849 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:55.349088 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:55.348964 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fmbfk" podUID="77822693-2416-4400-87cf-4619ac479e79" Jul 08 19:05:55.528648 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:55.528624 2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 08 19:05:55.744268 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:55.744235 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qm8rp"] Jul 08 19:05:55.744440 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:55.744367 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:55.744495 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:55.744467 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qm8rp" podUID="03cbe6c1-781c-4091-86ca-9de8256eda85" Jul 08 19:05:55.746601 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:55.746572 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fmbfk"] Jul 08 19:05:55.746703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:55.746671 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:55.746794 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:55.746769 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fmbfk" podUID="77822693-2416-4400-87cf-4619ac479e79" Jul 08 19:05:55.763286 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:55.763245 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2bqn6"] Jul 08 19:05:55.763455 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:55.763378 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:55.763552 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:55.763497 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bqn6" podUID="3981d8c6-c84c-439a-bf4b-d9834812c8bb" Jul 08 19:05:56.531016 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:56.530988 2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 08 19:05:57.262438 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:57.262395 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:05:57.349328 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:57.349247 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:57.349328 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:57.349276 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:57.349560 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:57.349266 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:57.349560 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:57.349376 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fmbfk" podUID="77822693-2416-4400-87cf-4619ac479e79" Jul 08 19:05:57.349560 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:57.349466 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qm8rp" podUID="03cbe6c1-781c-4091-86ca-9de8256eda85" Jul 08 19:05:57.349678 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:57.349586 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bqn6" podUID="3981d8c6-c84c-439a-bf4b-d9834812c8bb" Jul 08 19:05:57.535761 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:57.535724 2560 generic.go:356] "Generic (PLEG): container finished" podID="6349a4f6-297b-4718-9981-b1dbc8415f79" containerID="b302e6de8bcac2696c093520f443699684279b56dedc2c3dbaa2838af7ff77dc" exitCode=0 Jul 08 19:05:57.536146 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:57.535807 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wxhc" event={"ID":"6349a4f6-297b-4718-9981-b1dbc8415f79","Type":"ContainerDied","Data":"b302e6de8bcac2696c093520f443699684279b56dedc2c3dbaa2838af7ff77dc"} Jul 08 19:05:58.540547 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:58.540269 2560 generic.go:356] "Generic (PLEG): container finished" podID="6349a4f6-297b-4718-9981-b1dbc8415f79" containerID="eb12d70ae463e042ac61ff9da9fd475819b34eb7095bad6d79f37bbe23205359" exitCode=0 Jul 08 19:05:58.540547 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:58.540351 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wxhc" event={"ID":"6349a4f6-297b-4718-9981-b1dbc8415f79","Type":"ContainerDied","Data":"eb12d70ae463e042ac61ff9da9fd475819b34eb7095bad6d79f37bbe23205359"} Jul 08 19:05:59.348859 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:59.348833 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:05:59.348859 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:59.348853 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:59.349088 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:59.348927 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:05:59.349088 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:59.348927 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qm8rp" podUID="03cbe6c1-781c-4091-86ca-9de8256eda85" Jul 08 19:05:59.349088 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:59.349022 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fmbfk" podUID="77822693-2416-4400-87cf-4619ac479e79" Jul 08 19:05:59.349204 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:59.349123 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2bqn6" podUID="3981d8c6-c84c-439a-bf4b-d9834812c8bb" Jul 08 19:05:59.545013 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:59.544928 2560 generic.go:356] "Generic (PLEG): container finished" podID="6349a4f6-297b-4718-9981-b1dbc8415f79" containerID="ef362b8bfc65fe2b2cd8cad5b7b46a0c62704c509225402c4cbf5787861f402e" exitCode=0 Jul 08 19:05:59.545013 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:59.544967 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wxhc" event={"ID":"6349a4f6-297b-4718-9981-b1dbc8415f79","Type":"ContainerDied","Data":"ef362b8bfc65fe2b2cd8cad5b7b46a0c62704c509225402c4cbf5787861f402e"} Jul 08 19:05:59.914608 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:05:59.914574 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret\") pod \"global-pull-secret-syncer-fmbfk\" (UID: \"77822693-2416-4400-87cf-4619ac479e79\") " pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:05:59.914764 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:59.914705 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 19:05:59.914845 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:05:59.914769 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret podName:77822693-2416-4400-87cf-4619ac479e79 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:31.914751075 +0000 UTC m=+66.092885070 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret") pod "global-pull-secret-syncer-fmbfk" (UID: "77822693-2416-4400-87cf-4619ac479e79") : object "kube-system"/"original-pull-secret" not registered Jul 08 19:06:00.014965 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:00.014933 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs\") pod \"network-metrics-daemon-2bqn6\" (UID: \"3981d8c6-c84c-439a-bf4b-d9834812c8bb\") " pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:06:00.015119 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:00.015029 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:06:00.015119 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:00.015074 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs podName:3981d8c6-c84c-439a-bf4b-d9834812c8bb nodeName:}" failed. No retries permitted until 2026-07-08 19:06:32.015061285 +0000 UTC m=+66.193195269 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs") pod "network-metrics-daemon-2bqn6" (UID: "3981d8c6-c84c-439a-bf4b-d9834812c8bb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 19:06:00.115229 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:00.115186 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xp7d\" (UniqueName: \"kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d\") pod \"network-check-target-qm8rp\" (UID: \"03cbe6c1-781c-4091-86ca-9de8256eda85\") " pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:06:00.115396 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:00.115345 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 19:06:00.115396 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:00.115369 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 19:06:00.115556 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:00.115519 2560 projected.go:196] Error preparing data for projected volume kube-api-access-9xp7d for pod openshift-network-diagnostics/network-check-target-qm8rp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:06:00.115839 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:00.115807 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d podName:03cbe6c1-781c-4091-86ca-9de8256eda85 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:32.115783176 +0000 UTC m=+66.293917236 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9xp7d" (UniqueName: "kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d") pod "network-check-target-qm8rp" (UID: "03cbe6c1-781c-4091-86ca-9de8256eda85") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 19:06:01.132710 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.132682 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-38.ec2.internal" event="NodeReady" Jul 08 19:06:01.133184 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.132793 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 19:06:01.181101 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.181073 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-66755fc4fd-mrt27"] Jul 08 19:06:01.183008 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.182983 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.189846 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.189819 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 08 19:06:01.189979 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.189846 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 08 19:06:01.190891 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.190872 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mxm59\"" Jul 08 19:06:01.191246 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.191106 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 08 19:06:01.194869 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.194850 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 08 19:06:01.201544 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.200734 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-b42qc"] Jul 08 19:06:01.203211 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.203158 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bjxfb"] Jul 08 19:06:01.203302 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.203245 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b42qc" Jul 08 19:06:01.205186 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.205143 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-66755fc4fd-mrt27"] Jul 08 19:06:01.205284 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.205239 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:01.205854 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.205825 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 19:06:01.205953 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.205878 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 19:06:01.206016 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.205878 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 19:06:01.206121 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.206104 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qnk2b\"" Jul 08 19:06:01.210280 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.210260 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 19:06:01.211338 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.211319 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 19:06:01.211568 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.211549 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cxdf4\"" Jul 08 19:06:01.216218 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.216167 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b42qc"] Jul 08 19:06:01.222379 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.222296 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bjxfb"] Jul 08 19:06:01.328061 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.328024 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-certificates\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.328061 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.328063 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4ffcc504-f90b-4715-a91d-e3c436b5c822-trusted-ca\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.328297 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.328106 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4ffcc504-f90b-4715-a91d-e3c436b5c822-ca-trust-extracted\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.328297 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.328127 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-config-volume\") pod \"dns-default-bjxfb\" (UID: \"6e9b6b99-5daa-4d32-81fd-9b3aa823171b\") " pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:01.328297 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.328142 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-tmp-dir\") pod \"dns-default-bjxfb\" (UID: \"6e9b6b99-5daa-4d32-81fd-9b3aa823171b\") " pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:01.328297 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.328158 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clhmm\" (UniqueName: \"kubernetes.io/projected/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-kube-api-access-clhmm\") pod \"dns-default-bjxfb\" (UID: \"6e9b6b99-5daa-4d32-81fd-9b3aa823171b\") " pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:01.328297 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.328181 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6jzr\" (UniqueName: \"kubernetes.io/projected/987f6244-9238-41f0-9ba9-55b5815c995e-kube-api-access-p6jzr\") pod \"ingress-canary-b42qc\" (UID: \"987f6244-9238-41f0-9ba9-55b5815c995e\") " pod="openshift-ingress-canary/ingress-canary-b42qc" Jul 08 19:06:01.328297 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.328199 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-bound-sa-token\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.328297 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.328213 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-metrics-tls\") pod \"dns-default-bjxfb\" (UID: \"6e9b6b99-5daa-4d32-81fd-9b3aa823171b\") " pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:01.328297 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.328287 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-tls\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.328720 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.328337 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4ffcc504-f90b-4715-a91d-e3c436b5c822-installation-pull-secrets\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.328720 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.328357 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4ffcc504-f90b-4715-a91d-e3c436b5c822-image-registry-private-configuration\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.328720 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.328372 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/987f6244-9238-41f0-9ba9-55b5815c995e-cert\") pod \"ingress-canary-b42qc\" (UID: \"987f6244-9238-41f0-9ba9-55b5815c995e\") " pod="openshift-ingress-canary/ingress-canary-b42qc" Jul 08 19:06:01.328720 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.328392 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmhf2\" (UniqueName: \"kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-kube-api-access-lmhf2\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.348754 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.348725 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:06:01.348914 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.348771 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:06:01.348914 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.348725 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:06:01.352150 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.352129 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 19:06:01.352261 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.352202 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vfh9q\"" Jul 08 19:06:01.352784 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.352768 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 19:06:01.353250 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.353234 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 19:06:01.353681 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.353664 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fsv9h\"" Jul 08 19:06:01.353829 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.353813 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 19:06:01.428764 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.428729 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4ffcc504-f90b-4715-a91d-e3c436b5c822-installation-pull-secrets\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.428952 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.428776 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4ffcc504-f90b-4715-a91d-e3c436b5c822-image-registry-private-configuration\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.428952 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.428806 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/987f6244-9238-41f0-9ba9-55b5815c995e-cert\") pod \"ingress-canary-b42qc\" (UID: \"987f6244-9238-41f0-9ba9-55b5815c995e\") " pod="openshift-ingress-canary/ingress-canary-b42qc" Jul 08 19:06:01.428952 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.428850 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmhf2\" (UniqueName: \"kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-kube-api-access-lmhf2\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.428952 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.428906 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-certificates\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.428952 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.428928 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4ffcc504-f90b-4715-a91d-e3c436b5c822-trusted-ca\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.429184 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.428966 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4ffcc504-f90b-4715-a91d-e3c436b5c822-ca-trust-extracted\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.429184 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.428993 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-config-volume\") pod \"dns-default-bjxfb\" (UID: \"6e9b6b99-5daa-4d32-81fd-9b3aa823171b\") " pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:01.429184 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.429028 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-tmp-dir\") pod \"dns-default-bjxfb\" (UID: \"6e9b6b99-5daa-4d32-81fd-9b3aa823171b\") " pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:01.429184 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.429051 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clhmm\" (UniqueName: \"kubernetes.io/projected/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-kube-api-access-clhmm\") pod \"dns-default-bjxfb\" (UID: \"6e9b6b99-5daa-4d32-81fd-9b3aa823171b\") " pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:01.429184 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.429077 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6jzr\" (UniqueName: \"kubernetes.io/projected/987f6244-9238-41f0-9ba9-55b5815c995e-kube-api-access-p6jzr\") pod \"ingress-canary-b42qc\" (UID: \"987f6244-9238-41f0-9ba9-55b5815c995e\") " pod="openshift-ingress-canary/ingress-canary-b42qc" Jul 08 19:06:01.429184 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.429099 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-bound-sa-token\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.429184 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.429118 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-metrics-tls\") pod \"dns-default-bjxfb\" (UID: \"6e9b6b99-5daa-4d32-81fd-9b3aa823171b\") " pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:01.429184 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.429151 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-tls\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.429553 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:01.429292 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:06:01.429553 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:01.429307 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66755fc4fd-mrt27: secret "image-registry-tls" not found Jul 08 19:06:01.429553 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:01.429382 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-tls podName:4ffcc504-f90b-4715-a91d-e3c436b5c822 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:01.929362625 +0000 UTC m=+36.107496613 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-tls") pod "image-registry-66755fc4fd-mrt27" (UID: "4ffcc504-f90b-4715-a91d-e3c436b5c822") : secret "image-registry-tls" not found Jul 08 19:06:01.429958 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.429934 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4ffcc504-f90b-4715-a91d-e3c436b5c822-ca-trust-extracted\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.430156 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.430136 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-config-volume\") pod \"dns-default-bjxfb\" (UID: \"6e9b6b99-5daa-4d32-81fd-9b3aa823171b\") " pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:01.430340 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.430321 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-tmp-dir\") pod \"dns-default-bjxfb\" (UID: \"6e9b6b99-5daa-4d32-81fd-9b3aa823171b\") " pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:01.430452 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.430432 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-certificates\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.430708 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:01.430686 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:06:01.430807 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:01.430747 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-metrics-tls podName:6e9b6b99-5daa-4d32-81fd-9b3aa823171b nodeName:}" failed. No retries permitted until 2026-07-08 19:06:01.930731415 +0000 UTC m=+36.108865413 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-metrics-tls") pod "dns-default-bjxfb" (UID: "6e9b6b99-5daa-4d32-81fd-9b3aa823171b") : secret "dns-default-metrics-tls" not found Jul 08 19:06:01.431123 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.431102 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4ffcc504-f90b-4715-a91d-e3c436b5c822-trusted-ca\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.434018 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.433994 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4ffcc504-f90b-4715-a91d-e3c436b5c822-installation-pull-secrets\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.434125 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.434100 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/987f6244-9238-41f0-9ba9-55b5815c995e-cert\") pod \"ingress-canary-b42qc\" (UID: \"987f6244-9238-41f0-9ba9-55b5815c995e\") " pod="openshift-ingress-canary/ingress-canary-b42qc" Jul 08 19:06:01.434167 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.434127 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4ffcc504-f90b-4715-a91d-e3c436b5c822-image-registry-private-configuration\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.440196 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.440163 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clhmm\" (UniqueName: \"kubernetes.io/projected/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-kube-api-access-clhmm\") pod \"dns-default-bjxfb\" (UID: \"6e9b6b99-5daa-4d32-81fd-9b3aa823171b\") " pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:01.441344 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.441306 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-bound-sa-token\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.442513 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.442477 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmhf2\" (UniqueName: \"kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-kube-api-access-lmhf2\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.443727 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.443708 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6jzr\" (UniqueName: \"kubernetes.io/projected/987f6244-9238-41f0-9ba9-55b5815c995e-kube-api-access-p6jzr\") pod \"ingress-canary-b42qc\" (UID: \"987f6244-9238-41f0-9ba9-55b5815c995e\") " pod="openshift-ingress-canary/ingress-canary-b42qc" Jul 08 19:06:01.514914 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.514828 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b42qc" Jul 08 19:06:01.701726 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.701547 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b42qc"] Jul 08 19:06:01.709037 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:06:01.708440 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod987f6244_9238_41f0_9ba9_55b5815c995e.slice/crio-06b24bb3c76d382788d6c5402af269404895431c960a276fb624eaf209210a60 WatchSource:0}: Error finding container 06b24bb3c76d382788d6c5402af269404895431c960a276fb624eaf209210a60: Status 404 returned error can't find the container with id 06b24bb3c76d382788d6c5402af269404895431c960a276fb624eaf209210a60 Jul 08 19:06:01.933139 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.933106 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-metrics-tls\") pod \"dns-default-bjxfb\" (UID: \"6e9b6b99-5daa-4d32-81fd-9b3aa823171b\") " pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:01.933329 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:01.933155 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-tls\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:01.933329 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:01.933279 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:06:01.933329 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:01.933279 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:06:01.933482 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:01.933294 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66755fc4fd-mrt27: secret "image-registry-tls" not found Jul 08 19:06:01.933482 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:01.933373 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-metrics-tls podName:6e9b6b99-5daa-4d32-81fd-9b3aa823171b nodeName:}" failed. No retries permitted until 2026-07-08 19:06:02.933355869 +0000 UTC m=+37.111489854 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-metrics-tls") pod "dns-default-bjxfb" (UID: "6e9b6b99-5daa-4d32-81fd-9b3aa823171b") : secret "dns-default-metrics-tls" not found Jul 08 19:06:01.933482 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:01.933456 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-tls podName:4ffcc504-f90b-4715-a91d-e3c436b5c822 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:02.933437173 +0000 UTC m=+37.111571170 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-tls") pod "image-registry-66755fc4fd-mrt27" (UID: "4ffcc504-f90b-4715-a91d-e3c436b5c822") : secret "image-registry-tls" not found Jul 08 19:06:02.554478 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:02.554439 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b42qc" event={"ID":"987f6244-9238-41f0-9ba9-55b5815c995e","Type":"ContainerStarted","Data":"06b24bb3c76d382788d6c5402af269404895431c960a276fb624eaf209210a60"} Jul 08 19:06:02.941768 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:02.941723 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-metrics-tls\") pod \"dns-default-bjxfb\" (UID: \"6e9b6b99-5daa-4d32-81fd-9b3aa823171b\") " pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:02.941964 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:02.941786 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-tls\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:02.941964 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:02.941883 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:06:02.941964 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:02.941914 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:06:02.941964 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:02.941928 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66755fc4fd-mrt27: secret "image-registry-tls" not found Jul 08 19:06:02.941964 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:02.941955 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-metrics-tls podName:6e9b6b99-5daa-4d32-81fd-9b3aa823171b nodeName:}" failed. No retries permitted until 2026-07-08 19:06:04.941932396 +0000 UTC m=+39.120066397 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-metrics-tls") pod "dns-default-bjxfb" (UID: "6e9b6b99-5daa-4d32-81fd-9b3aa823171b") : secret "dns-default-metrics-tls" not found Jul 08 19:06:02.942217 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:02.941992 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-tls podName:4ffcc504-f90b-4715-a91d-e3c436b5c822 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:04.94197691 +0000 UTC m=+39.120110899 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-tls") pod "image-registry-66755fc4fd-mrt27" (UID: "4ffcc504-f90b-4715-a91d-e3c436b5c822") : secret "image-registry-tls" not found Jul 08 19:06:04.960612 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:04.960578 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-metrics-tls\") pod \"dns-default-bjxfb\" (UID: \"6e9b6b99-5daa-4d32-81fd-9b3aa823171b\") " pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:04.961110 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:04.960634 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-tls\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:04.961110 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:04.960786 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:06:04.961110 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:04.960804 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66755fc4fd-mrt27: secret "image-registry-tls" not found Jul 08 19:06:04.961110 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:04.960864 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-tls podName:4ffcc504-f90b-4715-a91d-e3c436b5c822 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:08.960843453 +0000 UTC m=+43.138977488 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-tls") pod "image-registry-66755fc4fd-mrt27" (UID: "4ffcc504-f90b-4715-a91d-e3c436b5c822") : secret "image-registry-tls" not found Jul 08 19:06:04.961328 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:04.961286 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:06:04.961388 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:04.961345 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-metrics-tls podName:6e9b6b99-5daa-4d32-81fd-9b3aa823171b nodeName:}" failed. No retries permitted until 2026-07-08 19:06:08.9613225 +0000 UTC m=+43.139456499 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-metrics-tls") pod "dns-default-bjxfb" (UID: "6e9b6b99-5daa-4d32-81fd-9b3aa823171b") : secret "dns-default-metrics-tls" not found Jul 08 19:06:05.564049 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:05.563966 2560 generic.go:356] "Generic (PLEG): container finished" podID="6349a4f6-297b-4718-9981-b1dbc8415f79" containerID="d7e4f6c50140f203dae130d2a57a24c2787795a0c6fffb3dc613b531d0af2a96" exitCode=0 Jul 08 19:06:05.564049 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:05.564029 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wxhc" event={"ID":"6349a4f6-297b-4718-9981-b1dbc8415f79","Type":"ContainerDied","Data":"d7e4f6c50140f203dae130d2a57a24c2787795a0c6fffb3dc613b531d0af2a96"} Jul 08 19:06:05.565483 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:05.565462 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b42qc" event={"ID":"987f6244-9238-41f0-9ba9-55b5815c995e","Type":"ContainerStarted","Data":"00879c442faf871d0f0c36dc2517ce10a2a1acce00a9462c301f0aea53a744fe"} Jul 08 19:06:05.604370 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:05.604325 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-b42qc" podStartSLOduration=1.064149795 podStartE2EDuration="4.604309038s" totalImagesPullingTime="3.540159243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:05.603154359 +0000 UTC m=+39.781288384" watchObservedRunningTime="2026-07-08 19:06:05.604309038 +0000 UTC m=+39.782443043" Jul 08 19:06:06.570703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:06.570518 2560 generic.go:356] "Generic (PLEG): container finished" podID="6349a4f6-297b-4718-9981-b1dbc8415f79" containerID="12dedbcda275d12964246deb1cce8e98d428dc3fc75427dd0de2262bd56e3ac7" exitCode=0 Jul 08 19:06:06.570703 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:06.570561 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wxhc" event={"ID":"6349a4f6-297b-4718-9981-b1dbc8415f79","Type":"ContainerDied","Data":"12dedbcda275d12964246deb1cce8e98d428dc3fc75427dd0de2262bd56e3ac7"} Jul 08 19:06:07.336155 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.336122 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xs97g"] Jul 08 19:06:07.364596 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.364570 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xs97g"] Jul 08 19:06:07.364735 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.364696 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:07.367253 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.367234 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vczqd\"" Jul 08 19:06:07.367601 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.367582 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 19:06:07.367768 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.367745 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 19:06:07.367995 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.367982 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 19:06:07.368217 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.368202 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 19:06:07.480004 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.479972 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/098a2451-3786-4094-9a48-19d8b3804a9f-crio-socket\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:07.480154 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.480032 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/098a2451-3786-4094-9a48-19d8b3804a9f-data-volume\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:07.480154 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.480075 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/098a2451-3786-4094-9a48-19d8b3804a9f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:07.480154 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.480104 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/098a2451-3786-4094-9a48-19d8b3804a9f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:07.480154 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.480135 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfbkq\" (UniqueName: \"kubernetes.io/projected/098a2451-3786-4094-9a48-19d8b3804a9f-kube-api-access-qfbkq\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:07.576351 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.576317 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5wxhc" event={"ID":"6349a4f6-297b-4718-9981-b1dbc8415f79","Type":"ContainerStarted","Data":"1db762d54e502fdd00410ffa978f9caafe53ba8a03653a78502fedddab22e093"} Jul 08 19:06:07.581054 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.581034 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/098a2451-3786-4094-9a48-19d8b3804a9f-data-volume\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:07.581136 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.581062 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/098a2451-3786-4094-9a48-19d8b3804a9f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:07.581136 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.581087 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/098a2451-3786-4094-9a48-19d8b3804a9f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:07.581224 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:07.581153 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:06:07.581224 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:07.581159 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:06:07.581224 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:07.581203 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/098a2451-3786-4094-9a48-19d8b3804a9f-insights-runtime-extractor-tls podName:098a2451-3786-4094-9a48-19d8b3804a9f nodeName:}" failed. No retries permitted until 2026-07-08 19:06:08.08119007 +0000 UTC m=+42.259324054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/098a2451-3786-4094-9a48-19d8b3804a9f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xs97g" (UID: "098a2451-3786-4094-9a48-19d8b3804a9f") : secret "insights-runtime-extractor-tls" not found Jul 08 19:06:07.581382 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.581228 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfbkq\" (UniqueName: \"kubernetes.io/projected/098a2451-3786-4094-9a48-19d8b3804a9f-kube-api-access-qfbkq\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:07.581382 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.581347 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/098a2451-3786-4094-9a48-19d8b3804a9f-crio-socket\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:07.581382 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:07.581365 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/098a2451-3786-4094-9a48-19d8b3804a9f-kube-rbac-proxy-cm podName:098a2451-3786-4094-9a48-19d8b3804a9f nodeName:}" failed. No retries permitted until 2026-07-08 19:06:08.081346748 +0000 UTC m=+42.259480735 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/098a2451-3786-4094-9a48-19d8b3804a9f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xs97g" (UID: "098a2451-3786-4094-9a48-19d8b3804a9f") : configmap "kube-rbac-proxy" not found Jul 08 19:06:07.581507 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.581385 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/098a2451-3786-4094-9a48-19d8b3804a9f-data-volume\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:07.581584 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.581568 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/098a2451-3786-4094-9a48-19d8b3804a9f-crio-socket\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:07.592953 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.592899 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfbkq\" (UniqueName: \"kubernetes.io/projected/098a2451-3786-4094-9a48-19d8b3804a9f-kube-api-access-qfbkq\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:07.603011 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.602970 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5wxhc" podStartSLOduration=8.08086676 podStartE2EDuration="41.602952535s" totalImagesPullingTime="33.522085775s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:07.601471939 +0000 UTC m=+41.779605956" watchObservedRunningTime="2026-07-08 19:06:07.602952535 +0000 UTC m=+41.781086544" Jul 08 19:06:07.692696 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.692662 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-fdtk5"] Jul 08 19:06:07.716784 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.716759 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-fdtk5"] Jul 08 19:06:07.716912 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.716853 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-fdtk5" Jul 08 19:06:07.719799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.719779 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 08 19:06:07.720342 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.720325 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 08 19:06:07.720429 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.720357 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-mfn98\"" Jul 08 19:06:07.884206 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.884115 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zngth\" (UniqueName: \"kubernetes.io/projected/9154ff70-49b0-4fbf-bdfe-cdc6d0bbad80-kube-api-access-zngth\") pod \"migrator-d57d58cc9-fdtk5\" (UID: \"9154ff70-49b0-4fbf-bdfe-cdc6d0bbad80\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-fdtk5" Jul 08 19:06:07.985008 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.984973 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zngth\" (UniqueName: \"kubernetes.io/projected/9154ff70-49b0-4fbf-bdfe-cdc6d0bbad80-kube-api-access-zngth\") pod \"migrator-d57d58cc9-fdtk5\" (UID: \"9154ff70-49b0-4fbf-bdfe-cdc6d0bbad80\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-fdtk5" Jul 08 19:06:07.994224 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:07.994195 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zngth\" (UniqueName: \"kubernetes.io/projected/9154ff70-49b0-4fbf-bdfe-cdc6d0bbad80-kube-api-access-zngth\") pod \"migrator-d57d58cc9-fdtk5\" (UID: \"9154ff70-49b0-4fbf-bdfe-cdc6d0bbad80\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-fdtk5" Jul 08 19:06:08.025457 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:08.025423 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-fdtk5" Jul 08 19:06:08.085747 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:08.085712 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/098a2451-3786-4094-9a48-19d8b3804a9f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:08.085899 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:08.085757 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/098a2451-3786-4094-9a48-19d8b3804a9f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:08.085899 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:08.085878 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:06:08.085976 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:08.085901 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:06:08.085976 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:08.085951 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/098a2451-3786-4094-9a48-19d8b3804a9f-insights-runtime-extractor-tls podName:098a2451-3786-4094-9a48-19d8b3804a9f nodeName:}" failed. No retries permitted until 2026-07-08 19:06:09.085937011 +0000 UTC m=+43.264071012 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/098a2451-3786-4094-9a48-19d8b3804a9f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xs97g" (UID: "098a2451-3786-4094-9a48-19d8b3804a9f") : secret "insights-runtime-extractor-tls" not found Jul 08 19:06:08.085976 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:08.085964 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/098a2451-3786-4094-9a48-19d8b3804a9f-kube-rbac-proxy-cm podName:098a2451-3786-4094-9a48-19d8b3804a9f nodeName:}" failed. No retries permitted until 2026-07-08 19:06:09.085957605 +0000 UTC m=+43.264091589 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/098a2451-3786-4094-9a48-19d8b3804a9f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xs97g" (UID: "098a2451-3786-4094-9a48-19d8b3804a9f") : configmap "kube-rbac-proxy" not found Jul 08 19:06:08.173499 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:08.173469 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-fdtk5"] Jul 08 19:06:08.176576 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:06:08.176546 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9154ff70_49b0_4fbf_bdfe_cdc6d0bbad80.slice/crio-fc345df66225bf926b5aa4367e4263fd40d44827c4883e06b74d88013adf273c WatchSource:0}: Error finding container fc345df66225bf926b5aa4367e4263fd40d44827c4883e06b74d88013adf273c: Status 404 returned error can't find the container with id fc345df66225bf926b5aa4367e4263fd40d44827c4883e06b74d88013adf273c Jul 08 19:06:08.579243 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:08.579159 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-fdtk5" event={"ID":"9154ff70-49b0-4fbf-bdfe-cdc6d0bbad80","Type":"ContainerStarted","Data":"fc345df66225bf926b5aa4367e4263fd40d44827c4883e06b74d88013adf273c"} Jul 08 19:06:08.994116 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:08.994081 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-metrics-tls\") pod \"dns-default-bjxfb\" (UID: \"6e9b6b99-5daa-4d32-81fd-9b3aa823171b\") " pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:08.994284 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:08.994125 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-tls\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:08.994284 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:08.994225 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:06:08.994284 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:08.994260 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:06:08.994284 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:08.994271 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66755fc4fd-mrt27: secret "image-registry-tls" not found Jul 08 19:06:08.994398 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:08.994291 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-metrics-tls podName:6e9b6b99-5daa-4d32-81fd-9b3aa823171b nodeName:}" failed. No retries permitted until 2026-07-08 19:06:16.994275926 +0000 UTC m=+51.172409910 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-metrics-tls") pod "dns-default-bjxfb" (UID: "6e9b6b99-5daa-4d32-81fd-9b3aa823171b") : secret "dns-default-metrics-tls" not found Jul 08 19:06:08.994398 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:08.994309 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-tls podName:4ffcc504-f90b-4715-a91d-e3c436b5c822 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:16.994297252 +0000 UTC m=+51.172431259 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-tls") pod "image-registry-66755fc4fd-mrt27" (UID: "4ffcc504-f90b-4715-a91d-e3c436b5c822") : secret "image-registry-tls" not found Jul 08 19:06:09.095062 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:09.095020 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/098a2451-3786-4094-9a48-19d8b3804a9f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:09.095251 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:09.095082 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/098a2451-3786-4094-9a48-19d8b3804a9f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:09.095251 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:09.095098 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:06:09.095251 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:09.095172 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/098a2451-3786-4094-9a48-19d8b3804a9f-kube-rbac-proxy-cm podName:098a2451-3786-4094-9a48-19d8b3804a9f nodeName:}" failed. No retries permitted until 2026-07-08 19:06:11.095157073 +0000 UTC m=+45.273291056 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/098a2451-3786-4094-9a48-19d8b3804a9f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xs97g" (UID: "098a2451-3786-4094-9a48-19d8b3804a9f") : configmap "kube-rbac-proxy" not found Jul 08 19:06:09.095251 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:09.095197 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:06:09.095251 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:09.095244 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/098a2451-3786-4094-9a48-19d8b3804a9f-insights-runtime-extractor-tls podName:098a2451-3786-4094-9a48-19d8b3804a9f nodeName:}" failed. No retries permitted until 2026-07-08 19:06:11.095230517 +0000 UTC m=+45.273364506 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/098a2451-3786-4094-9a48-19d8b3804a9f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xs97g" (UID: "098a2451-3786-4094-9a48-19d8b3804a9f") : secret "insights-runtime-extractor-tls" not found Jul 08 19:06:09.564207 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:09.564178 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sg8ph_a7d66e59-7d94-4a00-86f4-fbb8dfe65280/dns-node-resolver/0.log" Jul 08 19:06:10.560817 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:10.560742 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xs94k_04ab4794-25e2-4c82-9e44-b460a3993f26/node-ca/0.log" Jul 08 19:06:10.585308 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:10.585272 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-fdtk5" event={"ID":"9154ff70-49b0-4fbf-bdfe-cdc6d0bbad80","Type":"ContainerStarted","Data":"b558ef89cda50d5f75f9f7d76000d7a559aeab66a9f2002dfa5724a17bb7872b"} Jul 08 19:06:10.585443 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:10.585338 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-fdtk5" event={"ID":"9154ff70-49b0-4fbf-bdfe-cdc6d0bbad80","Type":"ContainerStarted","Data":"fde09ebe059c6228dff463370a7cb943fd8d599950176d29caf6d937cf15412f"} Jul 08 19:06:11.110152 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:11.110117 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/098a2451-3786-4094-9a48-19d8b3804a9f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:11.110152 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:11.110159 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/098a2451-3786-4094-9a48-19d8b3804a9f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:11.110417 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:11.110239 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:06:11.110417 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:11.110262 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:06:11.110417 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:11.110307 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/098a2451-3786-4094-9a48-19d8b3804a9f-insights-runtime-extractor-tls podName:098a2451-3786-4094-9a48-19d8b3804a9f nodeName:}" failed. No retries permitted until 2026-07-08 19:06:15.110292893 +0000 UTC m=+49.288426877 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/098a2451-3786-4094-9a48-19d8b3804a9f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xs97g" (UID: "098a2451-3786-4094-9a48-19d8b3804a9f") : secret "insights-runtime-extractor-tls" not found Jul 08 19:06:11.110417 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:11.110319 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/098a2451-3786-4094-9a48-19d8b3804a9f-kube-rbac-proxy-cm podName:098a2451-3786-4094-9a48-19d8b3804a9f nodeName:}" failed. No retries permitted until 2026-07-08 19:06:15.110313294 +0000 UTC m=+49.288447279 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/098a2451-3786-4094-9a48-19d8b3804a9f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xs97g" (UID: "098a2451-3786-4094-9a48-19d8b3804a9f") : configmap "kube-rbac-proxy" not found Jul 08 19:06:11.160208 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:11.160167 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b42qc_987f6244-9238-41f0-9ba9-55b5815c995e/serve-healthcheck-canary/0.log" Jul 08 19:06:15.141017 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:15.140987 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/098a2451-3786-4094-9a48-19d8b3804a9f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:15.141017 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:15.141030 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/098a2451-3786-4094-9a48-19d8b3804a9f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:15.141432 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:15.141117 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 08 19:06:15.141432 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:15.141135 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:06:15.141676 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:15.141661 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/098a2451-3786-4094-9a48-19d8b3804a9f-kube-rbac-proxy-cm podName:098a2451-3786-4094-9a48-19d8b3804a9f nodeName:}" failed. No retries permitted until 2026-07-08 19:06:23.14164461 +0000 UTC m=+57.319778598 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/098a2451-3786-4094-9a48-19d8b3804a9f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xs97g" (UID: "098a2451-3786-4094-9a48-19d8b3804a9f") : configmap "kube-rbac-proxy" not found Jul 08 19:06:15.141726 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:15.141687 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/098a2451-3786-4094-9a48-19d8b3804a9f-insights-runtime-extractor-tls podName:098a2451-3786-4094-9a48-19d8b3804a9f nodeName:}" failed. No retries permitted until 2026-07-08 19:06:23.141680259 +0000 UTC m=+57.319814243 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/098a2451-3786-4094-9a48-19d8b3804a9f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xs97g" (UID: "098a2451-3786-4094-9a48-19d8b3804a9f") : secret "insights-runtime-extractor-tls" not found Jul 08 19:06:17.056465 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:17.056424 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-metrics-tls\") pod \"dns-default-bjxfb\" (UID: \"6e9b6b99-5daa-4d32-81fd-9b3aa823171b\") " pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:17.057059 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:17.056479 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-tls\") pod \"image-registry-66755fc4fd-mrt27\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:17.057059 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:17.056607 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 08 19:06:17.057059 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:17.056623 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 08 19:06:17.057059 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:17.056640 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-66755fc4fd-mrt27: secret "image-registry-tls" not found Jul 08 19:06:17.057059 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:17.056704 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-tls podName:4ffcc504-f90b-4715-a91d-e3c436b5c822 nodeName:}" failed. No retries permitted until 2026-07-08 19:06:33.05668523 +0000 UTC m=+67.234819233 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-tls") pod "image-registry-66755fc4fd-mrt27" (UID: "4ffcc504-f90b-4715-a91d-e3c436b5c822") : secret "image-registry-tls" not found Jul 08 19:06:17.057059 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:17.056721 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-metrics-tls podName:6e9b6b99-5daa-4d32-81fd-9b3aa823171b nodeName:}" failed. No retries permitted until 2026-07-08 19:06:33.056712669 +0000 UTC m=+67.234846653 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-metrics-tls") pod "dns-default-bjxfb" (UID: "6e9b6b99-5daa-4d32-81fd-9b3aa823171b") : secret "dns-default-metrics-tls" not found Jul 08 19:06:23.206377 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:23.206341 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/098a2451-3786-4094-9a48-19d8b3804a9f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:23.206377 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:23.206386 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/098a2451-3786-4094-9a48-19d8b3804a9f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:23.206898 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:23.206498 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 08 19:06:23.206898 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:23.206576 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/098a2451-3786-4094-9a48-19d8b3804a9f-insights-runtime-extractor-tls podName:098a2451-3786-4094-9a48-19d8b3804a9f nodeName:}" failed. No retries permitted until 2026-07-08 19:06:39.206560949 +0000 UTC m=+73.384694934 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/098a2451-3786-4094-9a48-19d8b3804a9f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xs97g" (UID: "098a2451-3786-4094-9a48-19d8b3804a9f") : secret "insights-runtime-extractor-tls" not found Jul 08 19:06:23.206972 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:23.206893 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/098a2451-3786-4094-9a48-19d8b3804a9f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:27.556958 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:27.556930 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nklcn" Jul 08 19:06:27.589862 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:27.589809 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-fdtk5" podStartSLOduration=18.59044545 podStartE2EDuration="20.589791052s" totalImagesPullingTime="1.999345602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:10.60339876 +0000 UTC m=+44.781532766" watchObservedRunningTime="2026-07-08 19:06:27.589791052 +0000 UTC m=+61.767925072" Jul 08 19:06:31.585402 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.585347 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66755fc4fd-mrt27"] Jul 08 19:06:31.585801 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:31.585554 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" podUID="4ffcc504-f90b-4715-a91d-e3c436b5c822" Jul 08 19:06:31.637383 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.637357 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:31.646640 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.646614 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:31.671798 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.671771 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4ffcc504-f90b-4715-a91d-e3c436b5c822-installation-pull-secrets\") pod \"4ffcc504-f90b-4715-a91d-e3c436b5c822\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " Jul 08 19:06:31.671949 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.671810 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4ffcc504-f90b-4715-a91d-e3c436b5c822-ca-trust-extracted\") pod \"4ffcc504-f90b-4715-a91d-e3c436b5c822\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " Jul 08 19:06:31.671949 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.671846 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4ffcc504-f90b-4715-a91d-e3c436b5c822-trusted-ca\") pod \"4ffcc504-f90b-4715-a91d-e3c436b5c822\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " Jul 08 19:06:31.672066 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.671967 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4ffcc504-f90b-4715-a91d-e3c436b5c822-image-registry-private-configuration\") pod \"4ffcc504-f90b-4715-a91d-e3c436b5c822\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " Jul 08 19:06:31.672066 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.672008 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-bound-sa-token\") pod \"4ffcc504-f90b-4715-a91d-e3c436b5c822\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " Jul 08 19:06:31.672066 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.672024 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmhf2\" (UniqueName: \"kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-kube-api-access-lmhf2\") pod \"4ffcc504-f90b-4715-a91d-e3c436b5c822\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " Jul 08 19:06:31.672066 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.672046 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ffcc504-f90b-4715-a91d-e3c436b5c822-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4ffcc504-f90b-4715-a91d-e3c436b5c822" (UID: "4ffcc504-f90b-4715-a91d-e3c436b5c822"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:06:31.672263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.672063 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-certificates\") pod \"4ffcc504-f90b-4715-a91d-e3c436b5c822\" (UID: \"4ffcc504-f90b-4715-a91d-e3c436b5c822\") " Jul 08 19:06:31.672337 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.672312 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ffcc504-f90b-4715-a91d-e3c436b5c822-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4ffcc504-f90b-4715-a91d-e3c436b5c822" (UID: "4ffcc504-f90b-4715-a91d-e3c436b5c822"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:06:31.672389 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.672338 2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4ffcc504-f90b-4715-a91d-e3c436b5c822-ca-trust-extracted\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:06:31.672427 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.672378 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4ffcc504-f90b-4715-a91d-e3c436b5c822" (UID: "4ffcc504-f90b-4715-a91d-e3c436b5c822"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:06:31.674334 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.674301 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4ffcc504-f90b-4715-a91d-e3c436b5c822" (UID: "4ffcc504-f90b-4715-a91d-e3c436b5c822"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:06:31.674431 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.674340 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-kube-api-access-lmhf2" (OuterVolumeSpecName: "kube-api-access-lmhf2") pod "4ffcc504-f90b-4715-a91d-e3c436b5c822" (UID: "4ffcc504-f90b-4715-a91d-e3c436b5c822"). InnerVolumeSpecName "kube-api-access-lmhf2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:06:31.674473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.674461 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ffcc504-f90b-4715-a91d-e3c436b5c822-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4ffcc504-f90b-4715-a91d-e3c436b5c822" (UID: "4ffcc504-f90b-4715-a91d-e3c436b5c822"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:06:31.674519 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.674492 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ffcc504-f90b-4715-a91d-e3c436b5c822-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4ffcc504-f90b-4715-a91d-e3c436b5c822" (UID: "4ffcc504-f90b-4715-a91d-e3c436b5c822"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:06:31.772719 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.772687 2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-bound-sa-token\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:06:31.772719 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.772713 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmhf2\" (UniqueName: \"kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-kube-api-access-lmhf2\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:06:31.772719 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.772725 2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-certificates\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:06:31.772937 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.772735 2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4ffcc504-f90b-4715-a91d-e3c436b5c822-installation-pull-secrets\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:06:31.772937 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.772745 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4ffcc504-f90b-4715-a91d-e3c436b5c822-trusted-ca\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:06:31.772937 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.772753 2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4ffcc504-f90b-4715-a91d-e3c436b5c822-image-registry-private-configuration\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:06:31.973743 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.973709 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret\") pod \"global-pull-secret-syncer-fmbfk\" (UID: \"77822693-2416-4400-87cf-4619ac479e79\") " pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:06:31.977193 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.977176 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 19:06:31.986312 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:31.986285 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/77822693-2416-4400-87cf-4619ac479e79-original-pull-secret\") pod \"global-pull-secret-syncer-fmbfk\" (UID: \"77822693-2416-4400-87cf-4619ac479e79\") " pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:06:32.074105 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.074067 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs\") pod \"network-metrics-daemon-2bqn6\" (UID: \"3981d8c6-c84c-439a-bf4b-d9834812c8bb\") " pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:06:32.081861 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.081844 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 19:06:32.086452 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.086429 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3981d8c6-c84c-439a-bf4b-d9834812c8bb-metrics-certs\") pod \"network-metrics-daemon-2bqn6\" (UID: \"3981d8c6-c84c-439a-bf4b-d9834812c8bb\") " pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:06:32.174394 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.174361 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xp7d\" (UniqueName: \"kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d\") pod \"network-check-target-qm8rp\" (UID: \"03cbe6c1-781c-4091-86ca-9de8256eda85\") " pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:06:32.185458 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.185434 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 19:06:32.204265 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.204243 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 19:06:32.207737 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.207706 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xp7d\" (UniqueName: \"kubernetes.io/projected/03cbe6c1-781c-4091-86ca-9de8256eda85-kube-api-access-9xp7d\") pod \"network-check-target-qm8rp\" (UID: \"03cbe6c1-781c-4091-86ca-9de8256eda85\") " pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:06:32.257870 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.257796 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fmbfk" Jul 08 19:06:32.274118 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.274073 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-fsv9h\"" Jul 08 19:06:32.277706 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.277688 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vfh9q\"" Jul 08 19:06:32.284137 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.284118 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:06:32.288003 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.287974 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2bqn6" Jul 08 19:06:32.399130 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.399099 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fmbfk"] Jul 08 19:06:32.403311 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:06:32.403281 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77822693_2416_4400_87cf_4619ac479e79.slice/crio-a4bf6c676d249643f3f7207c39d8e3ff730279e782b5d1c47845dde035a38af9 WatchSource:0}: Error finding container a4bf6c676d249643f3f7207c39d8e3ff730279e782b5d1c47845dde035a38af9: Status 404 returned error can't find the container with id a4bf6c676d249643f3f7207c39d8e3ff730279e782b5d1c47845dde035a38af9 Jul 08 19:06:32.426108 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.426075 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qm8rp"] Jul 08 19:06:32.429902 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:06:32.429878 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03cbe6c1_781c_4091_86ca_9de8256eda85.slice/crio-46e4bdda0e7faa42f4f90562e3cd223fd38243915f34eb5c53d1fc010ce83a21 WatchSource:0}: Error finding container 46e4bdda0e7faa42f4f90562e3cd223fd38243915f34eb5c53d1fc010ce83a21: Status 404 returned error can't find the container with id 46e4bdda0e7faa42f4f90562e3cd223fd38243915f34eb5c53d1fc010ce83a21 Jul 08 19:06:32.447068 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.447046 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2bqn6"] Jul 08 19:06:32.449128 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:06:32.449103 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3981d8c6_c84c_439a_bf4b_d9834812c8bb.slice/crio-de0ba5f7fd32dc61e0d67029f8fbd9229d0250616618a4e3fb3723d8c8d16378 WatchSource:0}: Error finding container de0ba5f7fd32dc61e0d67029f8fbd9229d0250616618a4e3fb3723d8c8d16378: Status 404 returned error can't find the container with id de0ba5f7fd32dc61e0d67029f8fbd9229d0250616618a4e3fb3723d8c8d16378 Jul 08 19:06:32.641139 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.641108 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qm8rp" event={"ID":"03cbe6c1-781c-4091-86ca-9de8256eda85","Type":"ContainerStarted","Data":"46e4bdda0e7faa42f4f90562e3cd223fd38243915f34eb5c53d1fc010ce83a21"} Jul 08 19:06:32.642216 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.642198 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fmbfk" event={"ID":"77822693-2416-4400-87cf-4619ac479e79","Type":"ContainerStarted","Data":"a4bf6c676d249643f3f7207c39d8e3ff730279e782b5d1c47845dde035a38af9"} Jul 08 19:06:32.643265 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.643250 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-66755fc4fd-mrt27" Jul 08 19:06:32.643343 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.643282 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2bqn6" event={"ID":"3981d8c6-c84c-439a-bf4b-d9834812c8bb","Type":"ContainerStarted","Data":"de0ba5f7fd32dc61e0d67029f8fbd9229d0250616618a4e3fb3723d8c8d16378"} Jul 08 19:06:32.674510 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.674485 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-66755fc4fd-mrt27"] Jul 08 19:06:32.679109 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.679084 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-66755fc4fd-mrt27"] Jul 08 19:06:32.779898 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:32.779863 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4ffcc504-f90b-4715-a91d-e3c436b5c822-registry-tls\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:06:33.082320 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.082244 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-metrics-tls\") pod \"dns-default-bjxfb\" (UID: \"6e9b6b99-5daa-4d32-81fd-9b3aa823171b\") " pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:33.084643 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.084619 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/6e9b6b99-5daa-4d32-81fd-9b3aa823171b-metrics-tls\") pod \"dns-default-bjxfb\" (UID: \"6e9b6b99-5daa-4d32-81fd-9b3aa823171b\") " pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:33.325581 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.325557 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cxdf4\"" Jul 08 19:06:33.333572 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.333485 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:33.412039 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.412007 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jkccg"] Jul 08 19:06:33.417157 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.417126 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jkccg" Jul 08 19:06:33.420092 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.419490 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-6djrq\"" Jul 08 19:06:33.420092 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.419863 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 08 19:06:33.425585 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.425562 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jkccg"] Jul 08 19:06:33.485197 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.485103 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2ee4c41f-9323-452f-b3f4-90ddfeec954e-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-jkccg\" (UID: \"2ee4c41f-9323-452f-b3f4-90ddfeec954e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jkccg" Jul 08 19:06:33.587469 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.587382 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2ee4c41f-9323-452f-b3f4-90ddfeec954e-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-jkccg\" (UID: \"2ee4c41f-9323-452f-b3f4-90ddfeec954e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jkccg" Jul 08 19:06:33.590463 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.590433 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2ee4c41f-9323-452f-b3f4-90ddfeec954e-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-jkccg\" (UID: \"2ee4c41f-9323-452f-b3f4-90ddfeec954e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jkccg" Jul 08 19:06:33.597077 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.596981 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-gjhnf"] Jul 08 19:06:33.620681 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.620654 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-gjhnf"] Jul 08 19:06:33.620824 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.620768 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-gjhnf" Jul 08 19:06:33.623415 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.623388 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 19:06:33.623571 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.623471 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-cp5k8\"" Jul 08 19:06:33.623642 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.623392 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 19:06:33.675379 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.675348 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bjxfb"] Jul 08 19:06:33.680062 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:06:33.680037 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e9b6b99_5daa_4d32_81fd_9b3aa823171b.slice/crio-6b57e050d8d29bd106bc9a75db9a6ef53dd4fd56636eac8fb4027b277ff6497e WatchSource:0}: Error finding container 6b57e050d8d29bd106bc9a75db9a6ef53dd4fd56636eac8fb4027b277ff6497e: Status 404 returned error can't find the container with id 6b57e050d8d29bd106bc9a75db9a6ef53dd4fd56636eac8fb4027b277ff6497e Jul 08 19:06:33.688426 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.688398 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtqml\" (UniqueName: \"kubernetes.io/projected/ab32487f-34c8-4010-a1d7-aea7c4ab6378-kube-api-access-jtqml\") pod \"downloads-685bfb5f64-gjhnf\" (UID: \"ab32487f-34c8-4010-a1d7-aea7c4ab6378\") " pod="openshift-console/downloads-685bfb5f64-gjhnf" Jul 08 19:06:33.733313 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.733255 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jkccg" Jul 08 19:06:33.789123 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.789089 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtqml\" (UniqueName: \"kubernetes.io/projected/ab32487f-34c8-4010-a1d7-aea7c4ab6378-kube-api-access-jtqml\") pod \"downloads-685bfb5f64-gjhnf\" (UID: \"ab32487f-34c8-4010-a1d7-aea7c4ab6378\") " pod="openshift-console/downloads-685bfb5f64-gjhnf" Jul 08 19:06:33.800548 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.800473 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtqml\" (UniqueName: \"kubernetes.io/projected/ab32487f-34c8-4010-a1d7-aea7c4ab6378-kube-api-access-jtqml\") pod \"downloads-685bfb5f64-gjhnf\" (UID: \"ab32487f-34c8-4010-a1d7-aea7c4ab6378\") " pod="openshift-console/downloads-685bfb5f64-gjhnf" Jul 08 19:06:33.910517 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.910472 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jkccg"] Jul 08 19:06:33.913884 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:06:33.913857 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ee4c41f_9323_452f_b3f4_90ddfeec954e.slice/crio-a36258cc16543ff0f8b93ca71c02b3840a52ae47cf15b1950d15b384f6f80305 WatchSource:0}: Error finding container a36258cc16543ff0f8b93ca71c02b3840a52ae47cf15b1950d15b384f6f80305: Status 404 returned error can't find the container with id a36258cc16543ff0f8b93ca71c02b3840a52ae47cf15b1950d15b384f6f80305 Jul 08 19:06:33.932162 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:33.932136 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-gjhnf" Jul 08 19:06:34.117336 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:34.117265 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-gjhnf"] Jul 08 19:06:34.120170 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:06:34.120138 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab32487f_34c8_4010_a1d7_aea7c4ab6378.slice/crio-594e4d8122887c91f6900229206ce9ce94ed9a701c10c3203145f84db2b09155 WatchSource:0}: Error finding container 594e4d8122887c91f6900229206ce9ce94ed9a701c10c3203145f84db2b09155: Status 404 returned error can't find the container with id 594e4d8122887c91f6900229206ce9ce94ed9a701c10c3203145f84db2b09155 Jul 08 19:06:34.353567 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:34.353515 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ffcc504-f90b-4715-a91d-e3c436b5c822" path="/var/lib/kubelet/pods/4ffcc504-f90b-4715-a91d-e3c436b5c822/volumes" Jul 08 19:06:34.651091 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:34.651012 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jkccg" event={"ID":"2ee4c41f-9323-452f-b3f4-90ddfeec954e","Type":"ContainerStarted","Data":"a36258cc16543ff0f8b93ca71c02b3840a52ae47cf15b1950d15b384f6f80305"} Jul 08 19:06:34.652717 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:34.652688 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bjxfb" event={"ID":"6e9b6b99-5daa-4d32-81fd-9b3aa823171b","Type":"ContainerStarted","Data":"6b57e050d8d29bd106bc9a75db9a6ef53dd4fd56636eac8fb4027b277ff6497e"} Jul 08 19:06:34.655843 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:34.655820 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-gjhnf" event={"ID":"ab32487f-34c8-4010-a1d7-aea7c4ab6378","Type":"ContainerStarted","Data":"594e4d8122887c91f6900229206ce9ce94ed9a701c10c3203145f84db2b09155"} Jul 08 19:06:34.660977 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:34.660448 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2bqn6" event={"ID":"3981d8c6-c84c-439a-bf4b-d9834812c8bb","Type":"ContainerStarted","Data":"76d6f48bfc18def8524051c2bba075dd470c066d6b2771167aa2529e70c801f2"} Jul 08 19:06:34.660977 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:34.660497 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2bqn6" event={"ID":"3981d8c6-c84c-439a-bf4b-d9834812c8bb","Type":"ContainerStarted","Data":"6390fbb459bf20aa4ed55bad25ef661c62c10e674de81b8fa0f9f4d81bbcd046"} Jul 08 19:06:34.678160 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:34.678094 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2bqn6" podStartSLOduration=67.445430724 podStartE2EDuration="1m8.678075072s" totalImagesPullingTime="1.232644348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:34.677815392 +0000 UTC m=+68.855949399" watchObservedRunningTime="2026-07-08 19:06:34.678075072 +0000 UTC m=+68.856209082" Jul 08 19:06:39.234787 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:39.234748 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/098a2451-3786-4094-9a48-19d8b3804a9f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:39.237572 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:39.237515 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/098a2451-3786-4094-9a48-19d8b3804a9f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xs97g\" (UID: \"098a2451-3786-4094-9a48-19d8b3804a9f\") " pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:39.475681 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:39.475646 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-vczqd\"" Jul 08 19:06:39.483864 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:39.483683 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xs97g" Jul 08 19:06:39.683505 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:39.683440 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jkccg" event={"ID":"2ee4c41f-9323-452f-b3f4-90ddfeec954e","Type":"ContainerStarted","Data":"a58c3f7f067355de2136c4bfd2bb9371e88dd6e6c1dd6f31fbee7c5af0b8b2a1"} Jul 08 19:06:39.683960 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:39.683859 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jkccg" Jul 08 19:06:39.685558 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:39.685485 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bjxfb" event={"ID":"6e9b6b99-5daa-4d32-81fd-9b3aa823171b","Type":"ContainerStarted","Data":"9ead325b5d3f2118fa65a7edcc74cf3c4e22c3a72b8a1abe50597bdd0ce9de7f"} Jul 08 19:06:39.688609 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:39.688583 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qm8rp" event={"ID":"03cbe6c1-781c-4091-86ca-9de8256eda85","Type":"ContainerStarted","Data":"d7a44075f988fafb150502f8ccc7504aefa9a80ded3b4ac01a33c876f4c8c355"} Jul 08 19:06:39.689279 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:39.688963 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:06:39.689674 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:39.689634 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jkccg" Jul 08 19:06:39.691282 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:39.691259 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fmbfk" event={"ID":"77822693-2416-4400-87cf-4619ac479e79","Type":"ContainerStarted","Data":"90f68b1fb745be9d8d48ebc715c563b91b6a692d9ae65435080b573c4b203694"} Jul 08 19:06:39.696607 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:39.696435 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xs97g"] Jul 08 19:06:39.701751 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:06:39.701724 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod098a2451_3786_4094_9a48_19d8b3804a9f.slice/crio-4242a07f4aebee2bdf2376d7ff00bbdd657c9cbd16f2eef35c1710f0b3b4c5bb WatchSource:0}: Error finding container 4242a07f4aebee2bdf2376d7ff00bbdd657c9cbd16f2eef35c1710f0b3b4c5bb: Status 404 returned error can't find the container with id 4242a07f4aebee2bdf2376d7ff00bbdd657c9cbd16f2eef35c1710f0b3b4c5bb Jul 08 19:06:39.704643 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:39.704599 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-jkccg" podStartSLOduration=1.168411978 podStartE2EDuration="6.704582595s" totalImagesPullingTime="5.536170617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:39.703860283 +0000 UTC m=+73.881994290" watchObservedRunningTime="2026-07-08 19:06:39.704582595 +0000 UTC m=+73.882716603" Jul 08 19:06:39.724098 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:39.723920 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qm8rp" podStartSLOduration=66.686233133 podStartE2EDuration="1m13.723891525s" totalImagesPullingTime="7.037658392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:39.72330799 +0000 UTC m=+73.901441997" watchObservedRunningTime="2026-07-08 19:06:39.723891525 +0000 UTC m=+73.902025535" Jul 08 19:06:39.759919 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:39.759814 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fmbfk" podStartSLOduration=66.700900054 podStartE2EDuration="1m13.759790903s" totalImagesPullingTime="7.058890849s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:39.758967388 +0000 UTC m=+73.937101395" watchObservedRunningTime="2026-07-08 19:06:39.759790903 +0000 UTC m=+73.937924910" Jul 08 19:06:40.544838 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.544780 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7"] Jul 08 19:06:40.547573 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.547289 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7" Jul 08 19:06:40.551700 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.551512 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 08 19:06:40.551700 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.551578 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 19:06:40.552927 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.552399 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 08 19:06:40.552927 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.552634 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 19:06:40.552927 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.552790 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 19:06:40.553132 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.552992 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-7h72m\"" Jul 08 19:06:40.566282 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.564477 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7"] Jul 08 19:06:40.648352 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.648314 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-862bv\" (UniqueName: \"kubernetes.io/projected/34ed99b8-94b0-43a1-9895-a171af9a4e6f-kube-api-access-862bv\") pod \"prometheus-operator-567f7b8b7c-w5gd7\" (UID: \"34ed99b8-94b0-43a1-9895-a171af9a4e6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7" Jul 08 19:06:40.648586 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.648371 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34ed99b8-94b0-43a1-9895-a171af9a4e6f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-w5gd7\" (UID: \"34ed99b8-94b0-43a1-9895-a171af9a4e6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7" Jul 08 19:06:40.648586 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.648467 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34ed99b8-94b0-43a1-9895-a171af9a4e6f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-w5gd7\" (UID: \"34ed99b8-94b0-43a1-9895-a171af9a4e6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7" Jul 08 19:06:40.648586 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.648550 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/34ed99b8-94b0-43a1-9895-a171af9a4e6f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-w5gd7\" (UID: \"34ed99b8-94b0-43a1-9895-a171af9a4e6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7" Jul 08 19:06:40.701362 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.701320 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bjxfb" event={"ID":"6e9b6b99-5daa-4d32-81fd-9b3aa823171b","Type":"ContainerStarted","Data":"392977c69656beeadda396bf7dd03561f6d6c9c0f08d655ca8cd7f068a3c2d5e"} Jul 08 19:06:40.702202 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.702183 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:40.705415 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.705378 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xs97g" event={"ID":"098a2451-3786-4094-9a48-19d8b3804a9f","Type":"ContainerStarted","Data":"4242a07f4aebee2bdf2376d7ff00bbdd657c9cbd16f2eef35c1710f0b3b4c5bb"} Jul 08 19:06:40.724180 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.724102 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bjxfb" podStartSLOduration=33.955767763 podStartE2EDuration="39.724081146s" totalImagesPullingTime="5.768313383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:40.72305292 +0000 UTC m=+74.901186928" watchObservedRunningTime="2026-07-08 19:06:40.724081146 +0000 UTC m=+74.902215154" Jul 08 19:06:40.749346 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.749310 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-862bv\" (UniqueName: \"kubernetes.io/projected/34ed99b8-94b0-43a1-9895-a171af9a4e6f-kube-api-access-862bv\") pod \"prometheus-operator-567f7b8b7c-w5gd7\" (UID: \"34ed99b8-94b0-43a1-9895-a171af9a4e6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7" Jul 08 19:06:40.749518 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.749365 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34ed99b8-94b0-43a1-9895-a171af9a4e6f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-w5gd7\" (UID: \"34ed99b8-94b0-43a1-9895-a171af9a4e6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7" Jul 08 19:06:40.750058 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.749722 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34ed99b8-94b0-43a1-9895-a171af9a4e6f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-w5gd7\" (UID: \"34ed99b8-94b0-43a1-9895-a171af9a4e6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7" Jul 08 19:06:40.750058 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.749846 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/34ed99b8-94b0-43a1-9895-a171af9a4e6f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-w5gd7\" (UID: \"34ed99b8-94b0-43a1-9895-a171af9a4e6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7" Jul 08 19:06:40.750229 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:40.750208 2560 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 08 19:06:40.750301 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:40.750277 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34ed99b8-94b0-43a1-9895-a171af9a4e6f-prometheus-operator-tls podName:34ed99b8-94b0-43a1-9895-a171af9a4e6f nodeName:}" failed. No retries permitted until 2026-07-08 19:06:41.25025691 +0000 UTC m=+75.428390915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/34ed99b8-94b0-43a1-9895-a171af9a4e6f-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-w5gd7" (UID: "34ed99b8-94b0-43a1-9895-a171af9a4e6f") : secret "prometheus-operator-tls" not found Jul 08 19:06:40.750546 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.750498 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34ed99b8-94b0-43a1-9895-a171af9a4e6f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-w5gd7\" (UID: \"34ed99b8-94b0-43a1-9895-a171af9a4e6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7" Jul 08 19:06:40.758737 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.758676 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/34ed99b8-94b0-43a1-9895-a171af9a4e6f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-w5gd7\" (UID: \"34ed99b8-94b0-43a1-9895-a171af9a4e6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7" Jul 08 19:06:40.762614 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:40.762556 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-862bv\" (UniqueName: \"kubernetes.io/projected/34ed99b8-94b0-43a1-9895-a171af9a4e6f-kube-api-access-862bv\") pod \"prometheus-operator-567f7b8b7c-w5gd7\" (UID: \"34ed99b8-94b0-43a1-9895-a171af9a4e6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7" Jul 08 19:06:41.254300 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:41.254248 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/34ed99b8-94b0-43a1-9895-a171af9a4e6f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-w5gd7\" (UID: \"34ed99b8-94b0-43a1-9895-a171af9a4e6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7" Jul 08 19:06:41.259630 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:41.259571 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/34ed99b8-94b0-43a1-9895-a171af9a4e6f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-w5gd7\" (UID: \"34ed99b8-94b0-43a1-9895-a171af9a4e6f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7" Jul 08 19:06:41.462455 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:41.462414 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7" Jul 08 19:06:43.968906 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:43.968862 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7"] Jul 08 19:06:43.973054 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:06:43.972894 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34ed99b8_94b0_43a1_9895_a171af9a4e6f.slice/crio-ce452910e1179a5617f91e3b364d6b5fe6334bfd6ad3529f9d0a54ec419483d7 WatchSource:0}: Error finding container ce452910e1179a5617f91e3b364d6b5fe6334bfd6ad3529f9d0a54ec419483d7: Status 404 returned error can't find the container with id ce452910e1179a5617f91e3b364d6b5fe6334bfd6ad3529f9d0a54ec419483d7 Jul 08 19:06:44.721631 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:44.721585 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7" event={"ID":"34ed99b8-94b0-43a1-9895-a171af9a4e6f","Type":"ContainerStarted","Data":"ce452910e1179a5617f91e3b364d6b5fe6334bfd6ad3529f9d0a54ec419483d7"} Jul 08 19:06:44.723450 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:44.723412 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xs97g" event={"ID":"098a2451-3786-4094-9a48-19d8b3804a9f","Type":"ContainerStarted","Data":"5721ef7c4fd39dc880038eef6b09a4129ca3ed40b68e3da64bab626deabb76aa"} Jul 08 19:06:45.708108 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.708073 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-ff77bb95b-kd9g5"] Jul 08 19:06:45.710555 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.710376 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:45.712932 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.712900 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 19:06:45.718123 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.716452 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-76gvt\"" Jul 08 19:06:45.718123 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.717023 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 19:06:45.718123 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.717270 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 19:06:45.718123 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.717458 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 19:06:45.719197 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.718549 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 19:06:45.723149 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.723122 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-ff77bb95b-kd9g5"] Jul 08 19:06:45.725993 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.725969 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 19:06:45.731120 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.731073 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xs97g" event={"ID":"098a2451-3786-4094-9a48-19d8b3804a9f","Type":"ContainerStarted","Data":"d8160819a41b68b3854a492d79a8097f5c723eb6b2408dbc9330725f7c5f6d2c"} Jul 08 19:06:45.797770 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.797734 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b5f8012e-a53c-45b4-842d-43b8a512a548-console-oauth-config\") pod \"console-ff77bb95b-kd9g5\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:45.797947 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.797789 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5f8012e-a53c-45b4-842d-43b8a512a548-console-serving-cert\") pod \"console-ff77bb95b-kd9g5\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:45.797947 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.797822 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-trusted-ca-bundle\") pod \"console-ff77bb95b-kd9g5\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:45.797947 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.797862 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-console-config\") pod \"console-ff77bb95b-kd9g5\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:45.797947 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.797925 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-oauth-serving-cert\") pod \"console-ff77bb95b-kd9g5\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:45.798127 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.797951 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-service-ca\") pod \"console-ff77bb95b-kd9g5\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:45.798127 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.797996 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfnks\" (UniqueName: \"kubernetes.io/projected/b5f8012e-a53c-45b4-842d-43b8a512a548-kube-api-access-lfnks\") pod \"console-ff77bb95b-kd9g5\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:45.901599 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.899115 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-oauth-serving-cert\") pod \"console-ff77bb95b-kd9g5\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:45.901599 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.899168 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-service-ca\") pod \"console-ff77bb95b-kd9g5\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:45.901599 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.899206 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfnks\" (UniqueName: \"kubernetes.io/projected/b5f8012e-a53c-45b4-842d-43b8a512a548-kube-api-access-lfnks\") pod \"console-ff77bb95b-kd9g5\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:45.901599 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.899236 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b5f8012e-a53c-45b4-842d-43b8a512a548-console-oauth-config\") pod \"console-ff77bb95b-kd9g5\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:45.901599 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.899255 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5f8012e-a53c-45b4-842d-43b8a512a548-console-serving-cert\") pod \"console-ff77bb95b-kd9g5\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:45.901599 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.899270 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-trusted-ca-bundle\") pod \"console-ff77bb95b-kd9g5\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:45.901599 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.899301 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-console-config\") pod \"console-ff77bb95b-kd9g5\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:45.901599 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.900000 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-console-config\") pod \"console-ff77bb95b-kd9g5\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:45.901599 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.900556 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-oauth-serving-cert\") pod \"console-ff77bb95b-kd9g5\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:45.901599 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.900984 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-service-ca\") pod \"console-ff77bb95b-kd9g5\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:45.902688 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.902658 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-trusted-ca-bundle\") pod \"console-ff77bb95b-kd9g5\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:45.904337 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.904308 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5f8012e-a53c-45b4-842d-43b8a512a548-console-serving-cert\") pod \"console-ff77bb95b-kd9g5\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:45.905192 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.905167 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b5f8012e-a53c-45b4-842d-43b8a512a548-console-oauth-config\") pod \"console-ff77bb95b-kd9g5\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:45.910031 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:45.909986 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfnks\" (UniqueName: \"kubernetes.io/projected/b5f8012e-a53c-45b4-842d-43b8a512a548-kube-api-access-lfnks\") pod \"console-ff77bb95b-kd9g5\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:46.032691 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:46.032474 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:06:46.433183 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:46.433015 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-ff77bb95b-kd9g5"] Jul 08 19:06:46.738144 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:46.738101 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7" event={"ID":"34ed99b8-94b0-43a1-9895-a171af9a4e6f","Type":"ContainerStarted","Data":"6a4c6a21943f16d07d45e55a57b78d75a8e384978a13436ed8f60b00451abf1d"} Jul 08 19:06:46.738620 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:46.738173 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7" event={"ID":"34ed99b8-94b0-43a1-9895-a171af9a4e6f","Type":"ContainerStarted","Data":"068e33017f593adc7a4458f8158c596eeaffe8e82f4f84b9e22ff1ea44b005cb"} Jul 08 19:06:46.757697 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:46.757646 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-w5gd7" podStartSLOduration=4.435988084 podStartE2EDuration="6.757628409s" totalImagesPullingTime="2.321640325s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:46.756497051 +0000 UTC m=+80.934631091" watchObservedRunningTime="2026-07-08 19:06:46.757628409 +0000 UTC m=+80.935762417" Jul 08 19:06:47.682895 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:06:47.682858 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5f8012e_a53c_45b4_842d_43b8a512a548.slice/crio-386d243f621d8bd92e6ab2f7a9f009cb77755fcb8fc1450d5fb0a509b9e0a1dd WatchSource:0}: Error finding container 386d243f621d8bd92e6ab2f7a9f009cb77755fcb8fc1450d5fb0a509b9e0a1dd: Status 404 returned error can't find the container with id 386d243f621d8bd92e6ab2f7a9f009cb77755fcb8fc1450d5fb0a509b9e0a1dd Jul 08 19:06:47.741967 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:47.741940 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ff77bb95b-kd9g5" event={"ID":"b5f8012e-a53c-45b4-842d-43b8a512a548","Type":"ContainerStarted","Data":"386d243f621d8bd92e6ab2f7a9f009cb77755fcb8fc1450d5fb0a509b9e0a1dd"} Jul 08 19:06:48.748814 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:48.748769 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xs97g" event={"ID":"098a2451-3786-4094-9a48-19d8b3804a9f","Type":"ContainerStarted","Data":"0b19a02790bce7e2cc2970a3363bca559cc7edc09fee21ee4a189dcb1f72f2a4"} Jul 08 19:06:48.771992 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:48.771934 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xs97g" podStartSLOduration=33.914579383 podStartE2EDuration="41.771910882s" totalImagesPullingTime="7.857331499s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:06:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:48.771188772 +0000 UTC m=+82.949322782" watchObservedRunningTime="2026-07-08 19:06:48.771910882 +0000 UTC m=+82.950044888" Jul 08 19:06:48.869791 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:48.869755 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-r9427"] Jul 08 19:06:48.874470 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:48.874442 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:48.876829 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:48.876809 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 19:06:48.876988 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:48.876960 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fq6bn\"" Jul 08 19:06:48.877641 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:48.877620 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 19:06:48.877733 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:48.877676 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 19:06:48.929646 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:48.929598 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c1272e21-4e8e-476f-b367-093abb99cc1d-root\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:48.929646 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:48.929645 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1272e21-4e8e-476f-b367-093abb99cc1d-sys\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:48.929868 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:48.929692 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1272e21-4e8e-476f-b367-093abb99cc1d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:48.929868 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:48.929726 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1272e21-4e8e-476f-b367-093abb99cc1d-metrics-client-ca\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:48.929868 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:48.929775 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c1272e21-4e8e-476f-b367-093abb99cc1d-node-exporter-textfile\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:48.929868 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:48.929845 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c1272e21-4e8e-476f-b367-093abb99cc1d-node-exporter-accelerators-collector-config\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:48.930045 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:48.929896 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c1272e21-4e8e-476f-b367-093abb99cc1d-node-exporter-wtmp\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:48.930045 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:48.929922 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6f2w\" (UniqueName: \"kubernetes.io/projected/c1272e21-4e8e-476f-b367-093abb99cc1d-kube-api-access-m6f2w\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:48.930045 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:48.929993 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c1272e21-4e8e-476f-b367-093abb99cc1d-node-exporter-tls\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:49.031070 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:49.030992 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c1272e21-4e8e-476f-b367-093abb99cc1d-root\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:49.031070 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:49.031030 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1272e21-4e8e-476f-b367-093abb99cc1d-sys\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:49.031275 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:49.031096 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1272e21-4e8e-476f-b367-093abb99cc1d-sys\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:49.031275 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:49.031108 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c1272e21-4e8e-476f-b367-093abb99cc1d-root\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:49.031275 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:49.031133 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1272e21-4e8e-476f-b367-093abb99cc1d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:49.031275 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:49.031176 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1272e21-4e8e-476f-b367-093abb99cc1d-metrics-client-ca\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:49.031275 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:49.031203 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c1272e21-4e8e-476f-b367-093abb99cc1d-node-exporter-textfile\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:49.031275 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:49.031223 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c1272e21-4e8e-476f-b367-093abb99cc1d-node-exporter-accelerators-collector-config\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:49.031275 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:49.031265 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c1272e21-4e8e-476f-b367-093abb99cc1d-node-exporter-wtmp\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:49.031692 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:49.031288 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6f2w\" (UniqueName: \"kubernetes.io/projected/c1272e21-4e8e-476f-b367-093abb99cc1d-kube-api-access-m6f2w\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:49.031692 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:49.031324 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c1272e21-4e8e-476f-b367-093abb99cc1d-node-exporter-tls\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:49.031692 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:49.031443 2560 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 19:06:49.031692 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:06:49.031504 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1272e21-4e8e-476f-b367-093abb99cc1d-node-exporter-tls podName:c1272e21-4e8e-476f-b367-093abb99cc1d nodeName:}" failed. No retries permitted until 2026-07-08 19:06:49.531481547 +0000 UTC m=+83.709615544 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c1272e21-4e8e-476f-b367-093abb99cc1d-node-exporter-tls") pod "node-exporter-r9427" (UID: "c1272e21-4e8e-476f-b367-093abb99cc1d") : secret "node-exporter-tls" not found Jul 08 19:06:49.031692 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:49.031516 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c1272e21-4e8e-476f-b367-093abb99cc1d-node-exporter-wtmp\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:49.031938 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:49.031809 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1272e21-4e8e-476f-b367-093abb99cc1d-metrics-client-ca\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:49.031938 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:49.031926 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c1272e21-4e8e-476f-b367-093abb99cc1d-node-exporter-accelerators-collector-config\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:49.032032 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:49.031983 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c1272e21-4e8e-476f-b367-093abb99cc1d-node-exporter-textfile\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:49.033862 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:49.033841 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1272e21-4e8e-476f-b367-093abb99cc1d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:49.040629 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:49.040602 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6f2w\" (UniqueName: \"kubernetes.io/projected/c1272e21-4e8e-476f-b367-093abb99cc1d-kube-api-access-m6f2w\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:49.534896 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:49.534861 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c1272e21-4e8e-476f-b367-093abb99cc1d-node-exporter-tls\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:49.537899 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:49.537872 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c1272e21-4e8e-476f-b367-093abb99cc1d-node-exporter-tls\") pod \"node-exporter-r9427\" (UID: \"c1272e21-4e8e-476f-b367-093abb99cc1d\") " pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:49.784978 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:49.784896 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r9427" Jul 08 19:06:50.001352 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.001320 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 19:06:50.004988 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.004963 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.008919 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.008894 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 19:06:50.009063 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.009007 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 19:06:50.009063 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.009050 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-h5krj\"" Jul 08 19:06:50.009189 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.009177 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 19:06:50.009304 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.009281 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 19:06:50.009442 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.009424 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 19:06:50.009680 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.009665 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 19:06:50.009680 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.009674 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 19:06:50.009973 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.009957 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 19:06:50.010121 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.010103 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 19:06:50.033131 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.033100 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 19:06:50.139723 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.139689 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-web-config\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.139887 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.139737 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.139887 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.139773 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.139887 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.139814 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn722\" (UniqueName: \"kubernetes.io/projected/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-kube-api-access-dn722\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.139887 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.139864 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.140147 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.139892 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.140147 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.139937 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.140147 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.139971 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-config-volume\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.140147 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.139994 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.140147 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.140016 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.140147 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.140053 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-config-out\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.140147 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.140077 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.140147 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.140110 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.240716 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.240683 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-config-out\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.240716 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.240721 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.240980 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.240758 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.240980 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.240787 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-web-config\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.240980 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.240809 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.240980 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.240838 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.240980 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.240863 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn722\" (UniqueName: \"kubernetes.io/projected/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-kube-api-access-dn722\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.240980 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.240894 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.240980 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.240926 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.240980 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.240976 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.241356 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.241007 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-config-volume\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.241356 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.241043 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.241356 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.241067 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.241548 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.241514 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.242351 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.242323 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.242775 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.242708 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.244358 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.244333 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-config-out\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.244469 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.244337 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.245288 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.245251 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.245597 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.245577 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.245976 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.245955 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.246447 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.246149 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.246447 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.246371 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-web-config\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.246447 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.246416 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.248018 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.247994 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-config-volume\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.264297 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.264275 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn722\" (UniqueName: \"kubernetes.io/projected/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-kube-api-access-dn722\") pod \"alertmanager-main-0\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:50.316464 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:50.316430 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:06:51.714590 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:51.714550 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bjxfb" Jul 08 19:06:52.016667 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.016580 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6bbbd84965-f65nq"] Jul 08 19:06:52.020683 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.020655 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.050101 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.050073 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 08 19:06:52.051322 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.051303 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-g2s4v\"" Jul 08 19:06:52.051765 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.051744 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-1n7la6ud7fgu7\"" Jul 08 19:06:52.052074 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.052056 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 08 19:06:52.052155 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.052074 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 08 19:06:52.052155 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.052083 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 08 19:06:52.052270 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.052171 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 08 19:06:52.069282 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.069254 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6bbbd84965-f65nq"] Jul 08 19:06:52.159402 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.159368 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58f67a80-c6f3-4e56-a472-71946c9be733-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.159402 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.159402 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vppnh\" (UniqueName: \"kubernetes.io/projected/58f67a80-c6f3-4e56-a472-71946c9be733-kube-api-access-vppnh\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.159676 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.159429 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/58f67a80-c6f3-4e56-a472-71946c9be733-secret-grpc-tls\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.159676 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.159483 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58f67a80-c6f3-4e56-a472-71946c9be733-metrics-client-ca\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.159676 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.159538 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/58f67a80-c6f3-4e56-a472-71946c9be733-secret-thanos-querier-tls\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.159676 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.159568 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/58f67a80-c6f3-4e56-a472-71946c9be733-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.159676 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.159587 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/58f67a80-c6f3-4e56-a472-71946c9be733-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.159676 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.159622 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58f67a80-c6f3-4e56-a472-71946c9be733-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.260447 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.260408 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58f67a80-c6f3-4e56-a472-71946c9be733-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.260447 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.260450 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vppnh\" (UniqueName: \"kubernetes.io/projected/58f67a80-c6f3-4e56-a472-71946c9be733-kube-api-access-vppnh\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.260682 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.260478 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/58f67a80-c6f3-4e56-a472-71946c9be733-secret-grpc-tls\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.260682 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.260584 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58f67a80-c6f3-4e56-a472-71946c9be733-metrics-client-ca\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.260682 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.260650 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/58f67a80-c6f3-4e56-a472-71946c9be733-secret-thanos-querier-tls\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.260776 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.260703 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/58f67a80-c6f3-4e56-a472-71946c9be733-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.260776 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.260735 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/58f67a80-c6f3-4e56-a472-71946c9be733-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.260776 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.260762 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58f67a80-c6f3-4e56-a472-71946c9be733-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.261814 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.261717 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58f67a80-c6f3-4e56-a472-71946c9be733-metrics-client-ca\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.263993 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.263968 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/58f67a80-c6f3-4e56-a472-71946c9be733-secret-grpc-tls\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.264100 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.264034 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/58f67a80-c6f3-4e56-a472-71946c9be733-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.264452 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.264425 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58f67a80-c6f3-4e56-a472-71946c9be733-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.264584 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.264563 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58f67a80-c6f3-4e56-a472-71946c9be733-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.264640 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.264604 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/58f67a80-c6f3-4e56-a472-71946c9be733-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.264731 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.264712 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/58f67a80-c6f3-4e56-a472-71946c9be733-secret-thanos-querier-tls\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.293890 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.293782 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vppnh\" (UniqueName: \"kubernetes.io/projected/58f67a80-c6f3-4e56-a472-71946c9be733-kube-api-access-vppnh\") pod \"thanos-querier-6bbbd84965-f65nq\" (UID: \"58f67a80-c6f3-4e56-a472-71946c9be733\") " pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:52.333369 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:52.333331 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:06:53.284216 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.284182 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7df89d47dc-rkktq"] Jul 08 19:06:53.287541 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.287505 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.291805 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.291679 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-km5bm\"" Jul 08 19:06:53.291927 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.291911 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 08 19:06:53.292570 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.292351 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-anc7kq2kaq531\"" Jul 08 19:06:53.292570 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.292396 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 08 19:06:53.293569 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.293547 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 08 19:06:53.296681 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.296631 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 19:06:53.308623 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.308595 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7df89d47dc-rkktq"] Jul 08 19:06:53.373288 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.373258 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4ac7ccb3-0375-4b54-befa-a72aea0de69b-metrics-server-audit-profiles\") pod \"metrics-server-7df89d47dc-rkktq\" (UID: \"4ac7ccb3-0375-4b54-befa-a72aea0de69b\") " pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.373473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.373318 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ac7ccb3-0375-4b54-befa-a72aea0de69b-client-ca-bundle\") pod \"metrics-server-7df89d47dc-rkktq\" (UID: \"4ac7ccb3-0375-4b54-befa-a72aea0de69b\") " pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.373473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.373344 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ac7ccb3-0375-4b54-befa-a72aea0de69b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7df89d47dc-rkktq\" (UID: \"4ac7ccb3-0375-4b54-befa-a72aea0de69b\") " pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.373473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.373369 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4ac7ccb3-0375-4b54-befa-a72aea0de69b-audit-log\") pod \"metrics-server-7df89d47dc-rkktq\" (UID: \"4ac7ccb3-0375-4b54-befa-a72aea0de69b\") " pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.373473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.373435 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4ac7ccb3-0375-4b54-befa-a72aea0de69b-secret-metrics-server-tls\") pod \"metrics-server-7df89d47dc-rkktq\" (UID: \"4ac7ccb3-0375-4b54-befa-a72aea0de69b\") " pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.373473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.373457 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4ac7ccb3-0375-4b54-befa-a72aea0de69b-secret-metrics-server-client-certs\") pod \"metrics-server-7df89d47dc-rkktq\" (UID: \"4ac7ccb3-0375-4b54-befa-a72aea0de69b\") " pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.373719 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.373479 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9m46\" (UniqueName: \"kubernetes.io/projected/4ac7ccb3-0375-4b54-befa-a72aea0de69b-kube-api-access-p9m46\") pod \"metrics-server-7df89d47dc-rkktq\" (UID: \"4ac7ccb3-0375-4b54-befa-a72aea0de69b\") " pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.474395 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.474361 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4ac7ccb3-0375-4b54-befa-a72aea0de69b-audit-log\") pod \"metrics-server-7df89d47dc-rkktq\" (UID: \"4ac7ccb3-0375-4b54-befa-a72aea0de69b\") " pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.474599 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.474453 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4ac7ccb3-0375-4b54-befa-a72aea0de69b-secret-metrics-server-tls\") pod \"metrics-server-7df89d47dc-rkktq\" (UID: \"4ac7ccb3-0375-4b54-befa-a72aea0de69b\") " pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.474599 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.474506 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4ac7ccb3-0375-4b54-befa-a72aea0de69b-secret-metrics-server-client-certs\") pod \"metrics-server-7df89d47dc-rkktq\" (UID: \"4ac7ccb3-0375-4b54-befa-a72aea0de69b\") " pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.474599 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.474551 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9m46\" (UniqueName: \"kubernetes.io/projected/4ac7ccb3-0375-4b54-befa-a72aea0de69b-kube-api-access-p9m46\") pod \"metrics-server-7df89d47dc-rkktq\" (UID: \"4ac7ccb3-0375-4b54-befa-a72aea0de69b\") " pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.474599 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.474594 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4ac7ccb3-0375-4b54-befa-a72aea0de69b-metrics-server-audit-profiles\") pod \"metrics-server-7df89d47dc-rkktq\" (UID: \"4ac7ccb3-0375-4b54-befa-a72aea0de69b\") " pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.474764 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.474658 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ac7ccb3-0375-4b54-befa-a72aea0de69b-client-ca-bundle\") pod \"metrics-server-7df89d47dc-rkktq\" (UID: \"4ac7ccb3-0375-4b54-befa-a72aea0de69b\") " pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.474764 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.474695 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ac7ccb3-0375-4b54-befa-a72aea0de69b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7df89d47dc-rkktq\" (UID: \"4ac7ccb3-0375-4b54-befa-a72aea0de69b\") " pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.474825 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.474796 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4ac7ccb3-0375-4b54-befa-a72aea0de69b-audit-log\") pod \"metrics-server-7df89d47dc-rkktq\" (UID: \"4ac7ccb3-0375-4b54-befa-a72aea0de69b\") " pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.475344 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.475313 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ac7ccb3-0375-4b54-befa-a72aea0de69b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7df89d47dc-rkktq\" (UID: \"4ac7ccb3-0375-4b54-befa-a72aea0de69b\") " pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.475977 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.475897 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4ac7ccb3-0375-4b54-befa-a72aea0de69b-metrics-server-audit-profiles\") pod \"metrics-server-7df89d47dc-rkktq\" (UID: \"4ac7ccb3-0375-4b54-befa-a72aea0de69b\") " pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.477576 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.477554 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4ac7ccb3-0375-4b54-befa-a72aea0de69b-secret-metrics-server-tls\") pod \"metrics-server-7df89d47dc-rkktq\" (UID: \"4ac7ccb3-0375-4b54-befa-a72aea0de69b\") " pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.477705 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.477597 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4ac7ccb3-0375-4b54-befa-a72aea0de69b-secret-metrics-server-client-certs\") pod \"metrics-server-7df89d47dc-rkktq\" (UID: \"4ac7ccb3-0375-4b54-befa-a72aea0de69b\") " pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.477810 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.477790 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ac7ccb3-0375-4b54-befa-a72aea0de69b-client-ca-bundle\") pod \"metrics-server-7df89d47dc-rkktq\" (UID: \"4ac7ccb3-0375-4b54-befa-a72aea0de69b\") " pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.484239 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.484202 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9m46\" (UniqueName: \"kubernetes.io/projected/4ac7ccb3-0375-4b54-befa-a72aea0de69b-kube-api-access-p9m46\") pod \"metrics-server-7df89d47dc-rkktq\" (UID: \"4ac7ccb3-0375-4b54-befa-a72aea0de69b\") " pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:53.600054 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:53.599965 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:06:57.788187 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:06:57.788149 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1272e21_4e8e_476f_b367_093abb99cc1d.slice/crio-7def20b743ab5f6f0c00dfb7321ad229ebb31095facb02f07e32ff2e88b3fad3 WatchSource:0}: Error finding container 7def20b743ab5f6f0c00dfb7321ad229ebb31095facb02f07e32ff2e88b3fad3: Status 404 returned error can't find the container with id 7def20b743ab5f6f0c00dfb7321ad229ebb31095facb02f07e32ff2e88b3fad3 Jul 08 19:06:57.968330 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:57.968288 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 19:06:57.970723 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:06:57.970690 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d215f4f_ea16_449d_a0f7_ba0a0a1f7032.slice/crio-684c1f7caeb0992d6c907bf6065e35d8332d6b5bcb2a4e3728e0eb13280e5b16 WatchSource:0}: Error finding container 684c1f7caeb0992d6c907bf6065e35d8332d6b5bcb2a4e3728e0eb13280e5b16: Status 404 returned error can't find the container with id 684c1f7caeb0992d6c907bf6065e35d8332d6b5bcb2a4e3728e0eb13280e5b16 Jul 08 19:06:58.176672 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:58.176641 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7df89d47dc-rkktq"] Jul 08 19:06:58.179365 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:58.179327 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6bbbd84965-f65nq"] Jul 08 19:06:58.190613 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:06:58.190577 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58f67a80_c6f3_4e56_a472_71946c9be733.slice/crio-d8e6598a108bd10a97a7bd606f02a9382ce0bb38ac5c9f3f8406de7bee963c75 WatchSource:0}: Error finding container d8e6598a108bd10a97a7bd606f02a9382ce0bb38ac5c9f3f8406de7bee963c75: Status 404 returned error can't find the container with id d8e6598a108bd10a97a7bd606f02a9382ce0bb38ac5c9f3f8406de7bee963c75 Jul 08 19:06:58.789440 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:58.789392 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-gjhnf" event={"ID":"ab32487f-34c8-4010-a1d7-aea7c4ab6378","Type":"ContainerStarted","Data":"55fa31743effb07f112cab841e700ae83d6009e6afce433568cacfdc148ee010"} Jul 08 19:06:58.790422 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:58.790399 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-gjhnf" Jul 08 19:06:58.793552 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:58.792983 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ff77bb95b-kd9g5" event={"ID":"b5f8012e-a53c-45b4-842d-43b8a512a548","Type":"ContainerStarted","Data":"920a2b2007abb1e8a257327e1594be39c6b6b2cb494c6c3ca7dbd8edf237d0d3"} Jul 08 19:06:58.794625 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:58.794590 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032","Type":"ContainerStarted","Data":"684c1f7caeb0992d6c907bf6065e35d8332d6b5bcb2a4e3728e0eb13280e5b16"} Jul 08 19:06:58.796618 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:58.796571 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" event={"ID":"4ac7ccb3-0375-4b54-befa-a72aea0de69b","Type":"ContainerStarted","Data":"cbeda3c767ca073ea55528aa6ccc439e7973a663dc357fec4f7a995d4896261b"} Jul 08 19:06:58.798434 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:58.798378 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r9427" event={"ID":"c1272e21-4e8e-476f-b367-093abb99cc1d","Type":"ContainerStarted","Data":"7def20b743ab5f6f0c00dfb7321ad229ebb31095facb02f07e32ff2e88b3fad3"} Jul 08 19:06:58.799811 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:58.799782 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" event={"ID":"58f67a80-c6f3-4e56-a472-71946c9be733","Type":"ContainerStarted","Data":"d8e6598a108bd10a97a7bd606f02a9382ce0bb38ac5c9f3f8406de7bee963c75"} Jul 08 19:06:58.804393 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:58.804369 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-gjhnf" Jul 08 19:06:58.812678 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:58.812637 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-gjhnf" podStartSLOduration=2.125436557 podStartE2EDuration="25.812623946s" totalImagesPullingTime="23.687187389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:58.811619292 +0000 UTC m=+92.989753300" watchObservedRunningTime="2026-07-08 19:06:58.812623946 +0000 UTC m=+92.990757968" Jul 08 19:06:58.860722 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:58.860658 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-ff77bb95b-kd9g5" podStartSLOduration=3.752702941 podStartE2EDuration="13.860637911s" totalImagesPullingTime="10.10793497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:06:58.859142501 +0000 UTC m=+93.037276508" watchObservedRunningTime="2026-07-08 19:06:58.860637911 +0000 UTC m=+93.038771918" Jul 08 19:06:58.949853 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:58.949764 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-ff77bb95b-kd9g5"] Jul 08 19:06:58.993704 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:58.993664 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67875678f7-lhrpb"] Jul 08 19:06:59.013443 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.012806 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67875678f7-lhrpb"] Jul 08 19:06:59.013443 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.012924 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.126652 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.126561 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-oauth-serving-cert\") pod \"console-67875678f7-lhrpb\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.126819 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.126665 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-service-ca\") pod \"console-67875678f7-lhrpb\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.126819 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.126711 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-console-config\") pod \"console-67875678f7-lhrpb\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.126819 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.126734 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhxdm\" (UniqueName: \"kubernetes.io/projected/911f8ba6-0c9e-439d-b6d9-2bd635638042-kube-api-access-qhxdm\") pod \"console-67875678f7-lhrpb\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.126819 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.126766 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-trusted-ca-bundle\") pod \"console-67875678f7-lhrpb\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.126819 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.126795 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/911f8ba6-0c9e-439d-b6d9-2bd635638042-console-oauth-config\") pod \"console-67875678f7-lhrpb\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.127123 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.126820 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/911f8ba6-0c9e-439d-b6d9-2bd635638042-console-serving-cert\") pod \"console-67875678f7-lhrpb\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.228096 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.228058 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/911f8ba6-0c9e-439d-b6d9-2bd635638042-console-oauth-config\") pod \"console-67875678f7-lhrpb\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.228257 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.228110 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/911f8ba6-0c9e-439d-b6d9-2bd635638042-console-serving-cert\") pod \"console-67875678f7-lhrpb\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.228257 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.228149 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-oauth-serving-cert\") pod \"console-67875678f7-lhrpb\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.228257 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.228227 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-service-ca\") pod \"console-67875678f7-lhrpb\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.228257 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.228251 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-console-config\") pod \"console-67875678f7-lhrpb\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.228477 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.228278 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhxdm\" (UniqueName: \"kubernetes.io/projected/911f8ba6-0c9e-439d-b6d9-2bd635638042-kube-api-access-qhxdm\") pod \"console-67875678f7-lhrpb\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.228477 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.228320 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-trusted-ca-bundle\") pod \"console-67875678f7-lhrpb\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.229275 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.229253 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-trusted-ca-bundle\") pod \"console-67875678f7-lhrpb\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.230336 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.230312 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-service-ca\") pod \"console-67875678f7-lhrpb\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.232336 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.231406 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-console-config\") pod \"console-67875678f7-lhrpb\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.232336 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.232284 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-oauth-serving-cert\") pod \"console-67875678f7-lhrpb\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.233777 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.233732 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/911f8ba6-0c9e-439d-b6d9-2bd635638042-console-serving-cert\") pod \"console-67875678f7-lhrpb\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.240769 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.240745 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/911f8ba6-0c9e-439d-b6d9-2bd635638042-console-oauth-config\") pod \"console-67875678f7-lhrpb\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.244581 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.244516 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhxdm\" (UniqueName: \"kubernetes.io/projected/911f8ba6-0c9e-439d-b6d9-2bd635638042-kube-api-access-qhxdm\") pod \"console-67875678f7-lhrpb\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:06:59.327118 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:06:59.326839 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:07:00.003436 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:00.002970 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67875678f7-lhrpb"] Jul 08 19:07:00.814869 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:00.814008 2560 generic.go:356] "Generic (PLEG): container finished" podID="c1272e21-4e8e-476f-b367-093abb99cc1d" containerID="51410b9de920fba94a97ea80fb643210d69b8781656ad316a574c6b1fd4ed5b5" exitCode=0 Jul 08 19:07:00.814869 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:00.814308 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r9427" event={"ID":"c1272e21-4e8e-476f-b367-093abb99cc1d","Type":"ContainerDied","Data":"51410b9de920fba94a97ea80fb643210d69b8781656ad316a574c6b1fd4ed5b5"} Jul 08 19:07:00.816295 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:00.816265 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67875678f7-lhrpb" event={"ID":"911f8ba6-0c9e-439d-b6d9-2bd635638042","Type":"ContainerStarted","Data":"38fc5a102b35c8a1f5a146db867ecf403e64e6c45dc5fae79f25443669e25718"} Jul 08 19:07:00.816421 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:00.816321 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67875678f7-lhrpb" event={"ID":"911f8ba6-0c9e-439d-b6d9-2bd635638042","Type":"ContainerStarted","Data":"578efcfee4f88a081d8dba21cacce893c53fe4e6c99c6a5770ea6958fbe886f6"} Jul 08 19:07:00.818116 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:00.818091 2560 generic.go:356] "Generic (PLEG): container finished" podID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerID="d30f5770e55bfd01955466e93b89453c04341087edb89c708eb2bac24cca436e" exitCode=0 Jul 08 19:07:00.818215 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:00.818192 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032","Type":"ContainerDied","Data":"d30f5770e55bfd01955466e93b89453c04341087edb89c708eb2bac24cca436e"} Jul 08 19:07:01.825793 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:01.825760 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r9427" event={"ID":"c1272e21-4e8e-476f-b367-093abb99cc1d","Type":"ContainerStarted","Data":"b6d751a7d842644558b2daad5954ab9895538bf780a8fb05c5801f8f17a3942a"} Jul 08 19:07:01.902295 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:01.902238 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67875678f7-lhrpb" podStartSLOduration=3.902214453 podStartE2EDuration="3.902214453s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:07:01.900579976 +0000 UTC m=+96.078713983" watchObservedRunningTime="2026-07-08 19:07:01.902214453 +0000 UTC m=+96.080348459" Jul 08 19:07:02.836658 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:02.836565 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" event={"ID":"58f67a80-c6f3-4e56-a472-71946c9be733","Type":"ContainerStarted","Data":"ab24d1fdfeae5079eb7bc83f32abb840ca872009d589a70469b77a39a2f2baed"} Jul 08 19:07:02.837192 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:02.836690 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" event={"ID":"58f67a80-c6f3-4e56-a472-71946c9be733","Type":"ContainerStarted","Data":"e87d5910a549692e4fd8adadbeae227acf818b5990c8e68fab277b367cc2c61b"} Jul 08 19:07:02.837192 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:02.836719 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" event={"ID":"58f67a80-c6f3-4e56-a472-71946c9be733","Type":"ContainerStarted","Data":"decd808e33c641642035a8eb57915cef3d15653fa52a43c66dabc3111fee219a"} Jul 08 19:07:02.839381 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:02.839350 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" event={"ID":"4ac7ccb3-0375-4b54-befa-a72aea0de69b","Type":"ContainerStarted","Data":"98c48a6cbc9db77f5c78ac3335fffcd8a3f37740e32beed99eac16d32f8c29cc"} Jul 08 19:07:02.843651 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:02.843587 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r9427" event={"ID":"c1272e21-4e8e-476f-b367-093abb99cc1d","Type":"ContainerStarted","Data":"2214b42d3bb33b5aaa0d15dd97c0001ac77c49619a6f51ed13c3552fddd68eef"} Jul 08 19:07:02.888458 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:02.888367 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-r9427" podStartSLOduration=12.768517262 podStartE2EDuration="14.888349549s" totalImagesPullingTime="2.017434168s" totalInitContainerRuntime="102.398119ms" isStatefulPod=true podCreationTimestamp="2026-07-08 19:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:07:02.886882497 +0000 UTC m=+97.065016505" watchObservedRunningTime="2026-07-08 19:07:02.888349549 +0000 UTC m=+97.066483556" Jul 08 19:07:02.889298 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:02.889257 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" podStartSLOduration=6.246494905 podStartE2EDuration="9.889247926s" totalImagesPullingTime="3.642753021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:07:02.862547181 +0000 UTC m=+97.040681186" watchObservedRunningTime="2026-07-08 19:07:02.889247926 +0000 UTC m=+97.067381932" Jul 08 19:07:04.857745 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:04.857712 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" event={"ID":"58f67a80-c6f3-4e56-a472-71946c9be733","Type":"ContainerStarted","Data":"a7e35334b6873f4ace358168027672bab05119c4834b512c5aa1195501dc460d"} Jul 08 19:07:04.858133 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:04.857803 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" event={"ID":"58f67a80-c6f3-4e56-a472-71946c9be733","Type":"ContainerStarted","Data":"3c70bef354bfc302436c8e28aeca6e524ad81828e292349876a5641a86edb5ae"} Jul 08 19:07:04.860353 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:04.860327 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032","Type":"ContainerStarted","Data":"4a42d62d50b1d7826edc112de9014e7d9522c57909db34a8314f3cb44c211d6f"} Jul 08 19:07:04.860446 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:04.860404 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032","Type":"ContainerStarted","Data":"34c1a72e3be2f36fd24c49df268af305639a44f8bbd489e9fd8944772944ca70"} Jul 08 19:07:05.869059 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:05.869014 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032","Type":"ContainerStarted","Data":"97107613bdb5b8a919c2bce8d50214daeb16327fa78cc8ea9a729b40031bd915"} Jul 08 19:07:05.869648 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:05.869094 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032","Type":"ContainerStarted","Data":"06c3e2049515988466ccb1c66f899b0642fcecd33177e3230e6dc2404fb1d4db"} Jul 08 19:07:05.869648 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:05.869138 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032","Type":"ContainerStarted","Data":"97a155c81a448621b1930de4c8d501bbcf1288918a28c9a529f4c9867231064e"} Jul 08 19:07:05.869648 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:05.869181 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032","Type":"ContainerStarted","Data":"6bc130069dbb8512e8a864101cf23d5093411bc049aac1175770d46f44c20524"} Jul 08 19:07:05.872686 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:05.872657 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" event={"ID":"58f67a80-c6f3-4e56-a472-71946c9be733","Type":"ContainerStarted","Data":"f507d0d21495ef1a55d3bea586e56f8ab5a8f1bc272eed53f47ad33b8fee4a03"} Jul 08 19:07:05.872815 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:05.872799 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:07:05.901362 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:05.901307 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=11.33367515 podStartE2EDuration="16.901287355s" totalImagesPullingTime="5.55298749s" totalInitContainerRuntime="14.624715ms" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:07:05.899959229 +0000 UTC m=+100.078093236" watchObservedRunningTime="2026-07-08 19:07:05.901287355 +0000 UTC m=+100.079421362" Jul 08 19:07:06.033336 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:06.033297 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:07:09.327452 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:09.327408 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:07:09.327853 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:09.327515 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:07:09.332171 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:09.332148 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:07:09.352052 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:09.351980 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" podStartSLOduration=12.478203538 podStartE2EDuration="18.351966401s" totalImagesPullingTime="5.873762863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:06:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:07:05.928668881 +0000 UTC m=+100.106802889" watchObservedRunningTime="2026-07-08 19:07:09.351966401 +0000 UTC m=+103.530100409" Jul 08 19:07:09.896412 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:09.896373 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:07:11.713073 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:11.713036 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qm8rp" Jul 08 19:07:11.884460 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:11.884433 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6bbbd84965-f65nq" Jul 08 19:07:13.601085 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:13.601044 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:07:13.601496 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:13.601141 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:07:25.843540 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:25.843463 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-ff77bb95b-kd9g5" podUID="b5f8012e-a53c-45b4-842d-43b8a512a548" containerName="console" containerID="cri-o://920a2b2007abb1e8a257327e1594be39c6b6b2cb494c6c3ca7dbd8edf237d0d3" gracePeriod=15 Jul 08 19:07:26.116184 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.116115 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:07:26.171104 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.163638 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-oauth-serving-cert\") pod \"b5f8012e-a53c-45b4-842d-43b8a512a548\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " Jul 08 19:07:26.171104 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.163702 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-service-ca\") pod \"b5f8012e-a53c-45b4-842d-43b8a512a548\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " Jul 08 19:07:26.171104 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.163736 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfnks\" (UniqueName: \"kubernetes.io/projected/b5f8012e-a53c-45b4-842d-43b8a512a548-kube-api-access-lfnks\") pod \"b5f8012e-a53c-45b4-842d-43b8a512a548\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " Jul 08 19:07:26.171104 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.163769 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5f8012e-a53c-45b4-842d-43b8a512a548-console-serving-cert\") pod \"b5f8012e-a53c-45b4-842d-43b8a512a548\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " Jul 08 19:07:26.171104 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.163809 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-console-config\") pod \"b5f8012e-a53c-45b4-842d-43b8a512a548\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " Jul 08 19:07:26.171104 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.163844 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-trusted-ca-bundle\") pod \"b5f8012e-a53c-45b4-842d-43b8a512a548\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " Jul 08 19:07:26.171104 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.163920 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b5f8012e-a53c-45b4-842d-43b8a512a548-console-oauth-config\") pod \"b5f8012e-a53c-45b4-842d-43b8a512a548\" (UID: \"b5f8012e-a53c-45b4-842d-43b8a512a548\") " Jul 08 19:07:26.171800 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.171772 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b5f8012e-a53c-45b4-842d-43b8a512a548" (UID: "b5f8012e-a53c-45b4-842d-43b8a512a548"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:07:26.171887 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.171848 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-console-config" (OuterVolumeSpecName: "console-config") pod "b5f8012e-a53c-45b4-842d-43b8a512a548" (UID: "b5f8012e-a53c-45b4-842d-43b8a512a548"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:07:26.172015 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.171983 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-service-ca" (OuterVolumeSpecName: "service-ca") pod "b5f8012e-a53c-45b4-842d-43b8a512a548" (UID: "b5f8012e-a53c-45b4-842d-43b8a512a548"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:07:26.172239 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.172051 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5f8012e-a53c-45b4-842d-43b8a512a548-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b5f8012e-a53c-45b4-842d-43b8a512a548" (UID: "b5f8012e-a53c-45b4-842d-43b8a512a548"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:07:26.172239 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.172217 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b5f8012e-a53c-45b4-842d-43b8a512a548" (UID: "b5f8012e-a53c-45b4-842d-43b8a512a548"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:07:26.174318 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.174290 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5f8012e-a53c-45b4-842d-43b8a512a548-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b5f8012e-a53c-45b4-842d-43b8a512a548" (UID: "b5f8012e-a53c-45b4-842d-43b8a512a548"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:07:26.174840 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.174799 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5f8012e-a53c-45b4-842d-43b8a512a548-kube-api-access-lfnks" (OuterVolumeSpecName: "kube-api-access-lfnks") pod "b5f8012e-a53c-45b4-842d-43b8a512a548" (UID: "b5f8012e-a53c-45b4-842d-43b8a512a548"). InnerVolumeSpecName "kube-api-access-lfnks". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:07:26.264929 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.264901 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-console-config\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:07:26.264929 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.264923 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-trusted-ca-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:07:26.264929 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.264932 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b5f8012e-a53c-45b4-842d-43b8a512a548-console-oauth-config\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:07:26.265152 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.264941 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-oauth-serving-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:07:26.265152 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.264949 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b5f8012e-a53c-45b4-842d-43b8a512a548-service-ca\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:07:26.265152 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.264957 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfnks\" (UniqueName: \"kubernetes.io/projected/b5f8012e-a53c-45b4-842d-43b8a512a548-kube-api-access-lfnks\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:07:26.265152 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.264965 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b5f8012e-a53c-45b4-842d-43b8a512a548-console-serving-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:07:26.955600 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.955563 2560 generic.go:356] "Generic (PLEG): container finished" podID="b5f8012e-a53c-45b4-842d-43b8a512a548" containerID="920a2b2007abb1e8a257327e1594be39c6b6b2cb494c6c3ca7dbd8edf237d0d3" exitCode=0 Jul 08 19:07:26.955984 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.955626 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ff77bb95b-kd9g5" Jul 08 19:07:26.955984 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.955626 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ff77bb95b-kd9g5" event={"ID":"b5f8012e-a53c-45b4-842d-43b8a512a548","Type":"ContainerDied","Data":"920a2b2007abb1e8a257327e1594be39c6b6b2cb494c6c3ca7dbd8edf237d0d3"} Jul 08 19:07:26.955984 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.955707 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ff77bb95b-kd9g5" event={"ID":"b5f8012e-a53c-45b4-842d-43b8a512a548","Type":"ContainerDied","Data":"386d243f621d8bd92e6ab2f7a9f009cb77755fcb8fc1450d5fb0a509b9e0a1dd"} Jul 08 19:07:26.967445 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.967419 2560 scope.go:117] "RemoveContainer" containerID="920a2b2007abb1e8a257327e1594be39c6b6b2cb494c6c3ca7dbd8edf237d0d3" Jul 08 19:07:26.977130 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.977109 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-ff77bb95b-kd9g5"] Jul 08 19:07:26.980336 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.980315 2560 scope.go:117] "RemoveContainer" containerID="920a2b2007abb1e8a257327e1594be39c6b6b2cb494c6c3ca7dbd8edf237d0d3" Jul 08 19:07:26.980612 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:07:26.980591 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"920a2b2007abb1e8a257327e1594be39c6b6b2cb494c6c3ca7dbd8edf237d0d3\": container with ID starting with 920a2b2007abb1e8a257327e1594be39c6b6b2cb494c6c3ca7dbd8edf237d0d3 not found: ID does not exist" containerID="920a2b2007abb1e8a257327e1594be39c6b6b2cb494c6c3ca7dbd8edf237d0d3" Jul 08 19:07:26.980675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.980620 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"920a2b2007abb1e8a257327e1594be39c6b6b2cb494c6c3ca7dbd8edf237d0d3"} err="failed to get container status \"920a2b2007abb1e8a257327e1594be39c6b6b2cb494c6c3ca7dbd8edf237d0d3\": rpc error: code = NotFound desc = could not find container \"920a2b2007abb1e8a257327e1594be39c6b6b2cb494c6c3ca7dbd8edf237d0d3\": container with ID starting with 920a2b2007abb1e8a257327e1594be39c6b6b2cb494c6c3ca7dbd8edf237d0d3 not found: ID does not exist" Jul 08 19:07:26.983644 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:26.983625 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-ff77bb95b-kd9g5"] Jul 08 19:07:28.355490 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:28.355456 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5f8012e-a53c-45b4-842d-43b8a512a548" path="/var/lib/kubelet/pods/b5f8012e-a53c-45b4-842d-43b8a512a548/volumes" Jul 08 19:07:33.605363 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:33.605331 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:07:33.609100 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:07:33.609083 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7df89d47dc-rkktq" Jul 08 19:08:09.304577 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:09.304518 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 19:08:09.305059 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:09.304932 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="prom-label-proxy" containerID="cri-o://97107613bdb5b8a919c2bce8d50214daeb16327fa78cc8ea9a729b40031bd915" gracePeriod=120 Jul 08 19:08:09.305059 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:09.304954 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="kube-rbac-proxy" containerID="cri-o://97a155c81a448621b1930de4c8d501bbcf1288918a28c9a529f4c9867231064e" gracePeriod=120 Jul 08 19:08:09.305059 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:09.305006 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="config-reloader" containerID="cri-o://4a42d62d50b1d7826edc112de9014e7d9522c57909db34a8314f3cb44c211d6f" gracePeriod=120 Jul 08 19:08:09.305215 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:09.304913 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="alertmanager" containerID="cri-o://34c1a72e3be2f36fd24c49df268af305639a44f8bbd489e9fd8944772944ca70" gracePeriod=120 Jul 08 19:08:09.305215 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:09.305107 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="kube-rbac-proxy-metric" containerID="cri-o://06c3e2049515988466ccb1c66f899b0642fcecd33177e3230e6dc2404fb1d4db" gracePeriod=120 Jul 08 19:08:09.305215 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:09.304960 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="kube-rbac-proxy-web" containerID="cri-o://6bc130069dbb8512e8a864101cf23d5093411bc049aac1175770d46f44c20524" gracePeriod=120 Jul 08 19:08:10.119946 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.119909 2560 generic.go:356] "Generic (PLEG): container finished" podID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerID="97107613bdb5b8a919c2bce8d50214daeb16327fa78cc8ea9a729b40031bd915" exitCode=0 Jul 08 19:08:10.119946 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.119935 2560 generic.go:356] "Generic (PLEG): container finished" podID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerID="97a155c81a448621b1930de4c8d501bbcf1288918a28c9a529f4c9867231064e" exitCode=0 Jul 08 19:08:10.119946 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.119941 2560 generic.go:356] "Generic (PLEG): container finished" podID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerID="4a42d62d50b1d7826edc112de9014e7d9522c57909db34a8314f3cb44c211d6f" exitCode=0 Jul 08 19:08:10.119946 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.119947 2560 generic.go:356] "Generic (PLEG): container finished" podID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerID="34c1a72e3be2f36fd24c49df268af305639a44f8bbd489e9fd8944772944ca70" exitCode=0 Jul 08 19:08:10.120205 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.119982 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032","Type":"ContainerDied","Data":"97107613bdb5b8a919c2bce8d50214daeb16327fa78cc8ea9a729b40031bd915"} Jul 08 19:08:10.120205 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.120067 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032","Type":"ContainerDied","Data":"97a155c81a448621b1930de4c8d501bbcf1288918a28c9a529f4c9867231064e"} Jul 08 19:08:10.120205 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.120092 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032","Type":"ContainerDied","Data":"4a42d62d50b1d7826edc112de9014e7d9522c57909db34a8314f3cb44c211d6f"} Jul 08 19:08:10.120205 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.120118 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032","Type":"ContainerDied","Data":"34c1a72e3be2f36fd24c49df268af305639a44f8bbd489e9fd8944772944ca70"} Jul 08 19:08:10.548174 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.548152 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:10.614379 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.614340 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-main-tls\") pod \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " Jul 08 19:08:10.614571 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.614400 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-alertmanager-main-db\") pod \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " Jul 08 19:08:10.614571 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.614511 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-alertmanager-trusted-ca-bundle\") pod \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " Jul 08 19:08:10.614681 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.614582 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-kube-rbac-proxy-metric\") pod \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " Jul 08 19:08:10.614681 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.614627 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-metrics-client-ca\") pod \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " Jul 08 19:08:10.614681 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.614656 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn722\" (UniqueName: \"kubernetes.io/projected/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-kube-api-access-dn722\") pod \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " Jul 08 19:08:10.614831 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.614683 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-tls-assets\") pod \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " Jul 08 19:08:10.614831 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.614718 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-cluster-tls-config\") pod \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " Jul 08 19:08:10.614831 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.614739 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" (UID: "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:08:10.614831 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.614755 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-kube-rbac-proxy-web\") pod \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " Jul 08 19:08:10.614831 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.614802 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-config-out\") pod \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " Jul 08 19:08:10.615064 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.614836 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-config-volume\") pod \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " Jul 08 19:08:10.615064 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.614868 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-web-config\") pod \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " Jul 08 19:08:10.615064 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.614913 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-kube-rbac-proxy\") pod \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\" (UID: \"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032\") " Jul 08 19:08:10.615207 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.615150 2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-alertmanager-main-db\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:08:10.616635 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.616411 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" (UID: "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:08:10.616635 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.616576 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" (UID: "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:08:10.618113 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.618082 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" (UID: "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:08:10.618221 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.618162 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-config-out" (OuterVolumeSpecName: "config-out") pod "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" (UID: "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:08:10.618288 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.618172 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-kube-api-access-dn722" (OuterVolumeSpecName: "kube-api-access-dn722") pod "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" (UID: "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032"). InnerVolumeSpecName "kube-api-access-dn722". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:08:10.618468 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.618433 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" (UID: "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:08:10.618858 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.618815 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-config-volume" (OuterVolumeSpecName: "config-volume") pod "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" (UID: "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:08:10.618958 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.618882 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" (UID: "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:08:10.619423 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.619394 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" (UID: "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:08:10.619939 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.619906 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" (UID: "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:08:10.624886 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.624864 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" (UID: "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:08:10.629983 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.629962 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-web-config" (OuterVolumeSpecName: "web-config") pod "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" (UID: "3d215f4f-ea16-449d-a0f7-ba0a0a1f7032"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:08:10.715965 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.715936 2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-config-out\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:08:10.715965 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.715960 2560 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-config-volume\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:08:10.715965 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.715970 2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-web-config\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:08:10.716182 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.715980 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:08:10.716182 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.715990 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-main-tls\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:08:10.716182 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.716000 2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:08:10.716182 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.716008 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:08:10.716182 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.716017 2560 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-metrics-client-ca\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:08:10.716182 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.716025 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn722\" (UniqueName: \"kubernetes.io/projected/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-kube-api-access-dn722\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:08:10.716182 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.716032 2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-tls-assets\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:08:10.716182 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.716040 2560 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-cluster-tls-config\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:08:10.716182 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:10.716048 2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:08:11.126709 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.126631 2560 generic.go:356] "Generic (PLEG): container finished" podID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerID="06c3e2049515988466ccb1c66f899b0642fcecd33177e3230e6dc2404fb1d4db" exitCode=0 Jul 08 19:08:11.126709 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.126656 2560 generic.go:356] "Generic (PLEG): container finished" podID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerID="6bc130069dbb8512e8a864101cf23d5093411bc049aac1175770d46f44c20524" exitCode=0 Jul 08 19:08:11.126875 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.126714 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032","Type":"ContainerDied","Data":"06c3e2049515988466ccb1c66f899b0642fcecd33177e3230e6dc2404fb1d4db"} Jul 08 19:08:11.126875 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.126721 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.126875 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.126815 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032","Type":"ContainerDied","Data":"6bc130069dbb8512e8a864101cf23d5093411bc049aac1175770d46f44c20524"} Jul 08 19:08:11.126875 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.126845 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3d215f4f-ea16-449d-a0f7-ba0a0a1f7032","Type":"ContainerDied","Data":"684c1f7caeb0992d6c907bf6065e35d8332d6b5bcb2a4e3728e0eb13280e5b16"} Jul 08 19:08:11.127003 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.126880 2560 scope.go:117] "RemoveContainer" containerID="97107613bdb5b8a919c2bce8d50214daeb16327fa78cc8ea9a729b40031bd915" Jul 08 19:08:11.134980 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.134962 2560 scope.go:117] "RemoveContainer" containerID="06c3e2049515988466ccb1c66f899b0642fcecd33177e3230e6dc2404fb1d4db" Jul 08 19:08:11.141867 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.141845 2560 scope.go:117] "RemoveContainer" containerID="97a155c81a448621b1930de4c8d501bbcf1288918a28c9a529f4c9867231064e" Jul 08 19:08:11.148267 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.148248 2560 scope.go:117] "RemoveContainer" containerID="6bc130069dbb8512e8a864101cf23d5093411bc049aac1175770d46f44c20524" Jul 08 19:08:11.152902 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.152880 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 19:08:11.156008 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.155991 2560 scope.go:117] "RemoveContainer" containerID="4a42d62d50b1d7826edc112de9014e7d9522c57909db34a8314f3cb44c211d6f" Jul 08 19:08:11.156393 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.156373 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 19:08:11.162512 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.162492 2560 scope.go:117] "RemoveContainer" containerID="34c1a72e3be2f36fd24c49df268af305639a44f8bbd489e9fd8944772944ca70" Jul 08 19:08:11.169696 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.169678 2560 scope.go:117] "RemoveContainer" containerID="d30f5770e55bfd01955466e93b89453c04341087edb89c708eb2bac24cca436e" Jul 08 19:08:11.176692 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.176676 2560 scope.go:117] "RemoveContainer" containerID="97107613bdb5b8a919c2bce8d50214daeb16327fa78cc8ea9a729b40031bd915" Jul 08 19:08:11.176946 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:08:11.176930 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97107613bdb5b8a919c2bce8d50214daeb16327fa78cc8ea9a729b40031bd915\": container with ID starting with 97107613bdb5b8a919c2bce8d50214daeb16327fa78cc8ea9a729b40031bd915 not found: ID does not exist" containerID="97107613bdb5b8a919c2bce8d50214daeb16327fa78cc8ea9a729b40031bd915" Jul 08 19:08:11.177014 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.176958 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97107613bdb5b8a919c2bce8d50214daeb16327fa78cc8ea9a729b40031bd915"} err="failed to get container status \"97107613bdb5b8a919c2bce8d50214daeb16327fa78cc8ea9a729b40031bd915\": rpc error: code = NotFound desc = could not find container \"97107613bdb5b8a919c2bce8d50214daeb16327fa78cc8ea9a729b40031bd915\": container with ID starting with 97107613bdb5b8a919c2bce8d50214daeb16327fa78cc8ea9a729b40031bd915 not found: ID does not exist" Jul 08 19:08:11.177014 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.176984 2560 scope.go:117] "RemoveContainer" containerID="06c3e2049515988466ccb1c66f899b0642fcecd33177e3230e6dc2404fb1d4db" Jul 08 19:08:11.177224 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:08:11.177205 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06c3e2049515988466ccb1c66f899b0642fcecd33177e3230e6dc2404fb1d4db\": container with ID starting with 06c3e2049515988466ccb1c66f899b0642fcecd33177e3230e6dc2404fb1d4db not found: ID does not exist" containerID="06c3e2049515988466ccb1c66f899b0642fcecd33177e3230e6dc2404fb1d4db" Jul 08 19:08:11.177263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.177231 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06c3e2049515988466ccb1c66f899b0642fcecd33177e3230e6dc2404fb1d4db"} err="failed to get container status \"06c3e2049515988466ccb1c66f899b0642fcecd33177e3230e6dc2404fb1d4db\": rpc error: code = NotFound desc = could not find container \"06c3e2049515988466ccb1c66f899b0642fcecd33177e3230e6dc2404fb1d4db\": container with ID starting with 06c3e2049515988466ccb1c66f899b0642fcecd33177e3230e6dc2404fb1d4db not found: ID does not exist" Jul 08 19:08:11.177263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.177247 2560 scope.go:117] "RemoveContainer" containerID="97a155c81a448621b1930de4c8d501bbcf1288918a28c9a529f4c9867231064e" Jul 08 19:08:11.177469 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:08:11.177453 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97a155c81a448621b1930de4c8d501bbcf1288918a28c9a529f4c9867231064e\": container with ID starting with 97a155c81a448621b1930de4c8d501bbcf1288918a28c9a529f4c9867231064e not found: ID does not exist" containerID="97a155c81a448621b1930de4c8d501bbcf1288918a28c9a529f4c9867231064e" Jul 08 19:08:11.177545 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.177488 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a155c81a448621b1930de4c8d501bbcf1288918a28c9a529f4c9867231064e"} err="failed to get container status \"97a155c81a448621b1930de4c8d501bbcf1288918a28c9a529f4c9867231064e\": rpc error: code = NotFound desc = could not find container \"97a155c81a448621b1930de4c8d501bbcf1288918a28c9a529f4c9867231064e\": container with ID starting with 97a155c81a448621b1930de4c8d501bbcf1288918a28c9a529f4c9867231064e not found: ID does not exist" Jul 08 19:08:11.177545 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.177509 2560 scope.go:117] "RemoveContainer" containerID="6bc130069dbb8512e8a864101cf23d5093411bc049aac1175770d46f44c20524" Jul 08 19:08:11.177752 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:08:11.177735 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bc130069dbb8512e8a864101cf23d5093411bc049aac1175770d46f44c20524\": container with ID starting with 6bc130069dbb8512e8a864101cf23d5093411bc049aac1175770d46f44c20524 not found: ID does not exist" containerID="6bc130069dbb8512e8a864101cf23d5093411bc049aac1175770d46f44c20524" Jul 08 19:08:11.177799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.177757 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bc130069dbb8512e8a864101cf23d5093411bc049aac1175770d46f44c20524"} err="failed to get container status \"6bc130069dbb8512e8a864101cf23d5093411bc049aac1175770d46f44c20524\": rpc error: code = NotFound desc = could not find container \"6bc130069dbb8512e8a864101cf23d5093411bc049aac1175770d46f44c20524\": container with ID starting with 6bc130069dbb8512e8a864101cf23d5093411bc049aac1175770d46f44c20524 not found: ID does not exist" Jul 08 19:08:11.177799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.177780 2560 scope.go:117] "RemoveContainer" containerID="4a42d62d50b1d7826edc112de9014e7d9522c57909db34a8314f3cb44c211d6f" Jul 08 19:08:11.178029 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:08:11.178013 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a42d62d50b1d7826edc112de9014e7d9522c57909db34a8314f3cb44c211d6f\": container with ID starting with 4a42d62d50b1d7826edc112de9014e7d9522c57909db34a8314f3cb44c211d6f not found: ID does not exist" containerID="4a42d62d50b1d7826edc112de9014e7d9522c57909db34a8314f3cb44c211d6f" Jul 08 19:08:11.178081 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.178032 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a42d62d50b1d7826edc112de9014e7d9522c57909db34a8314f3cb44c211d6f"} err="failed to get container status \"4a42d62d50b1d7826edc112de9014e7d9522c57909db34a8314f3cb44c211d6f\": rpc error: code = NotFound desc = could not find container \"4a42d62d50b1d7826edc112de9014e7d9522c57909db34a8314f3cb44c211d6f\": container with ID starting with 4a42d62d50b1d7826edc112de9014e7d9522c57909db34a8314f3cb44c211d6f not found: ID does not exist" Jul 08 19:08:11.178081 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.178044 2560 scope.go:117] "RemoveContainer" containerID="34c1a72e3be2f36fd24c49df268af305639a44f8bbd489e9fd8944772944ca70" Jul 08 19:08:11.178275 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:08:11.178258 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34c1a72e3be2f36fd24c49df268af305639a44f8bbd489e9fd8944772944ca70\": container with ID starting with 34c1a72e3be2f36fd24c49df268af305639a44f8bbd489e9fd8944772944ca70 not found: ID does not exist" containerID="34c1a72e3be2f36fd24c49df268af305639a44f8bbd489e9fd8944772944ca70" Jul 08 19:08:11.178324 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.178279 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34c1a72e3be2f36fd24c49df268af305639a44f8bbd489e9fd8944772944ca70"} err="failed to get container status \"34c1a72e3be2f36fd24c49df268af305639a44f8bbd489e9fd8944772944ca70\": rpc error: code = NotFound desc = could not find container \"34c1a72e3be2f36fd24c49df268af305639a44f8bbd489e9fd8944772944ca70\": container with ID starting with 34c1a72e3be2f36fd24c49df268af305639a44f8bbd489e9fd8944772944ca70 not found: ID does not exist" Jul 08 19:08:11.178324 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.178293 2560 scope.go:117] "RemoveContainer" containerID="d30f5770e55bfd01955466e93b89453c04341087edb89c708eb2bac24cca436e" Jul 08 19:08:11.178491 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:08:11.178473 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d30f5770e55bfd01955466e93b89453c04341087edb89c708eb2bac24cca436e\": container with ID starting with d30f5770e55bfd01955466e93b89453c04341087edb89c708eb2bac24cca436e not found: ID does not exist" containerID="d30f5770e55bfd01955466e93b89453c04341087edb89c708eb2bac24cca436e" Jul 08 19:08:11.178541 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.178493 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d30f5770e55bfd01955466e93b89453c04341087edb89c708eb2bac24cca436e"} err="failed to get container status \"d30f5770e55bfd01955466e93b89453c04341087edb89c708eb2bac24cca436e\": rpc error: code = NotFound desc = could not find container \"d30f5770e55bfd01955466e93b89453c04341087edb89c708eb2bac24cca436e\": container with ID starting with d30f5770e55bfd01955466e93b89453c04341087edb89c708eb2bac24cca436e not found: ID does not exist" Jul 08 19:08:11.178541 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.178505 2560 scope.go:117] "RemoveContainer" containerID="97107613bdb5b8a919c2bce8d50214daeb16327fa78cc8ea9a729b40031bd915" Jul 08 19:08:11.178746 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.178730 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97107613bdb5b8a919c2bce8d50214daeb16327fa78cc8ea9a729b40031bd915"} err="failed to get container status \"97107613bdb5b8a919c2bce8d50214daeb16327fa78cc8ea9a729b40031bd915\": rpc error: code = NotFound desc = could not find container \"97107613bdb5b8a919c2bce8d50214daeb16327fa78cc8ea9a729b40031bd915\": container with ID starting with 97107613bdb5b8a919c2bce8d50214daeb16327fa78cc8ea9a729b40031bd915 not found: ID does not exist" Jul 08 19:08:11.178746 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.178746 2560 scope.go:117] "RemoveContainer" containerID="06c3e2049515988466ccb1c66f899b0642fcecd33177e3230e6dc2404fb1d4db" Jul 08 19:08:11.179008 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.178987 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06c3e2049515988466ccb1c66f899b0642fcecd33177e3230e6dc2404fb1d4db"} err="failed to get container status \"06c3e2049515988466ccb1c66f899b0642fcecd33177e3230e6dc2404fb1d4db\": rpc error: code = NotFound desc = could not find container \"06c3e2049515988466ccb1c66f899b0642fcecd33177e3230e6dc2404fb1d4db\": container with ID starting with 06c3e2049515988466ccb1c66f899b0642fcecd33177e3230e6dc2404fb1d4db not found: ID does not exist" Jul 08 19:08:11.179097 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.179009 2560 scope.go:117] "RemoveContainer" containerID="97a155c81a448621b1930de4c8d501bbcf1288918a28c9a529f4c9867231064e" Jul 08 19:08:11.179486 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.179402 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a155c81a448621b1930de4c8d501bbcf1288918a28c9a529f4c9867231064e"} err="failed to get container status \"97a155c81a448621b1930de4c8d501bbcf1288918a28c9a529f4c9867231064e\": rpc error: code = NotFound desc = could not find container \"97a155c81a448621b1930de4c8d501bbcf1288918a28c9a529f4c9867231064e\": container with ID starting with 97a155c81a448621b1930de4c8d501bbcf1288918a28c9a529f4c9867231064e not found: ID does not exist" Jul 08 19:08:11.179486 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.179426 2560 scope.go:117] "RemoveContainer" containerID="6bc130069dbb8512e8a864101cf23d5093411bc049aac1175770d46f44c20524" Jul 08 19:08:11.179885 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.179804 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bc130069dbb8512e8a864101cf23d5093411bc049aac1175770d46f44c20524"} err="failed to get container status \"6bc130069dbb8512e8a864101cf23d5093411bc049aac1175770d46f44c20524\": rpc error: code = NotFound desc = could not find container \"6bc130069dbb8512e8a864101cf23d5093411bc049aac1175770d46f44c20524\": container with ID starting with 6bc130069dbb8512e8a864101cf23d5093411bc049aac1175770d46f44c20524 not found: ID does not exist" Jul 08 19:08:11.179885 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.179827 2560 scope.go:117] "RemoveContainer" containerID="4a42d62d50b1d7826edc112de9014e7d9522c57909db34a8314f3cb44c211d6f" Jul 08 19:08:11.180157 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.180085 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a42d62d50b1d7826edc112de9014e7d9522c57909db34a8314f3cb44c211d6f"} err="failed to get container status \"4a42d62d50b1d7826edc112de9014e7d9522c57909db34a8314f3cb44c211d6f\": rpc error: code = NotFound desc = could not find container \"4a42d62d50b1d7826edc112de9014e7d9522c57909db34a8314f3cb44c211d6f\": container with ID starting with 4a42d62d50b1d7826edc112de9014e7d9522c57909db34a8314f3cb44c211d6f not found: ID does not exist" Jul 08 19:08:11.180157 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.180105 2560 scope.go:117] "RemoveContainer" containerID="34c1a72e3be2f36fd24c49df268af305639a44f8bbd489e9fd8944772944ca70" Jul 08 19:08:11.180422 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.180333 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34c1a72e3be2f36fd24c49df268af305639a44f8bbd489e9fd8944772944ca70"} err="failed to get container status \"34c1a72e3be2f36fd24c49df268af305639a44f8bbd489e9fd8944772944ca70\": rpc error: code = NotFound desc = could not find container \"34c1a72e3be2f36fd24c49df268af305639a44f8bbd489e9fd8944772944ca70\": container with ID starting with 34c1a72e3be2f36fd24c49df268af305639a44f8bbd489e9fd8944772944ca70 not found: ID does not exist" Jul 08 19:08:11.180422 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.180352 2560 scope.go:117] "RemoveContainer" containerID="d30f5770e55bfd01955466e93b89453c04341087edb89c708eb2bac24cca436e" Jul 08 19:08:11.180679 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.180649 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d30f5770e55bfd01955466e93b89453c04341087edb89c708eb2bac24cca436e"} err="failed to get container status \"d30f5770e55bfd01955466e93b89453c04341087edb89c708eb2bac24cca436e\": rpc error: code = NotFound desc = could not find container \"d30f5770e55bfd01955466e93b89453c04341087edb89c708eb2bac24cca436e\": container with ID starting with d30f5770e55bfd01955466e93b89453c04341087edb89c708eb2bac24cca436e not found: ID does not exist" Jul 08 19:08:11.183304 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183285 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 19:08:11.183558 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183545 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="alertmanager" Jul 08 19:08:11.183605 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183559 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="alertmanager" Jul 08 19:08:11.183605 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183568 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="config-reloader" Jul 08 19:08:11.183605 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183574 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="config-reloader" Jul 08 19:08:11.183605 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183581 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="prom-label-proxy" Jul 08 19:08:11.183605 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183587 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="prom-label-proxy" Jul 08 19:08:11.183605 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183598 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="init-config-reloader" Jul 08 19:08:11.183605 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183604 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="init-config-reloader" Jul 08 19:08:11.183802 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183610 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="kube-rbac-proxy" Jul 08 19:08:11.183802 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183615 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="kube-rbac-proxy" Jul 08 19:08:11.183802 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183623 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f8012e-a53c-45b4-842d-43b8a512a548" containerName="console" Jul 08 19:08:11.183802 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183628 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f8012e-a53c-45b4-842d-43b8a512a548" containerName="console" Jul 08 19:08:11.183802 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183633 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="kube-rbac-proxy-web" Jul 08 19:08:11.183802 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183638 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="kube-rbac-proxy-web" Jul 08 19:08:11.183802 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183651 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="kube-rbac-proxy-metric" Jul 08 19:08:11.183802 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183656 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="kube-rbac-proxy-metric" Jul 08 19:08:11.183802 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183695 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="prom-label-proxy" Jul 08 19:08:11.183802 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183703 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5f8012e-a53c-45b4-842d-43b8a512a548" containerName="console" Jul 08 19:08:11.183802 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183709 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="kube-rbac-proxy-metric" Jul 08 19:08:11.183802 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183715 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="alertmanager" Jul 08 19:08:11.183802 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183719 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="kube-rbac-proxy" Jul 08 19:08:11.183802 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183725 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="config-reloader" Jul 08 19:08:11.183802 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.183730 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" containerName="kube-rbac-proxy-web" Jul 08 19:08:11.187648 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.187629 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.190232 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.190212 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 19:08:11.190334 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.190236 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 19:08:11.190334 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.190310 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 19:08:11.190334 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.190315 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 19:08:11.190334 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.190330 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 19:08:11.190550 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.190329 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 19:08:11.190550 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.190429 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 19:08:11.190550 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.190478 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 19:08:11.190843 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.190826 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-h5krj\"" Jul 08 19:08:11.196899 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.196879 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 19:08:11.201301 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.201280 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 19:08:11.221329 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.221201 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e23d33c-4709-4320-8359-dea6305b5869-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.221607 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.221275 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7e23d33c-4709-4320-8359-dea6305b5869-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.221701 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.221612 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e23d33c-4709-4320-8359-dea6305b5869-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.221701 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.221679 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e23d33c-4709-4320-8359-dea6305b5869-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.221810 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.221714 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7e23d33c-4709-4320-8359-dea6305b5869-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.221810 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.221746 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e23d33c-4709-4320-8359-dea6305b5869-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.221810 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.221783 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7e23d33c-4709-4320-8359-dea6305b5869-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.221964 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.221809 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e23d33c-4709-4320-8359-dea6305b5869-config-out\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.221964 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.221851 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddz4h\" (UniqueName: \"kubernetes.io/projected/7e23d33c-4709-4320-8359-dea6305b5869-kube-api-access-ddz4h\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.221964 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.221877 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7e23d33c-4709-4320-8359-dea6305b5869-config-volume\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.221964 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.221912 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e23d33c-4709-4320-8359-dea6305b5869-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.221964 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.221934 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e23d33c-4709-4320-8359-dea6305b5869-web-config\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.221964 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.221957 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7e23d33c-4709-4320-8359-dea6305b5869-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.323191 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.323152 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7e23d33c-4709-4320-8359-dea6305b5869-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.323191 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.323198 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e23d33c-4709-4320-8359-dea6305b5869-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.323723 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.323227 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e23d33c-4709-4320-8359-dea6305b5869-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.323723 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.323252 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7e23d33c-4709-4320-8359-dea6305b5869-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.323723 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.323274 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e23d33c-4709-4320-8359-dea6305b5869-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.323723 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.323298 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7e23d33c-4709-4320-8359-dea6305b5869-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.323723 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.323319 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e23d33c-4709-4320-8359-dea6305b5869-config-out\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.323723 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.323357 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddz4h\" (UniqueName: \"kubernetes.io/projected/7e23d33c-4709-4320-8359-dea6305b5869-kube-api-access-ddz4h\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.323723 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.323382 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7e23d33c-4709-4320-8359-dea6305b5869-config-volume\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.323723 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.323413 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e23d33c-4709-4320-8359-dea6305b5869-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.323723 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.323436 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e23d33c-4709-4320-8359-dea6305b5869-web-config\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.323723 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.323462 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7e23d33c-4709-4320-8359-dea6305b5869-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.323723 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.323492 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e23d33c-4709-4320-8359-dea6305b5869-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.324347 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.324323 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e23d33c-4709-4320-8359-dea6305b5869-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.325104 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.324700 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7e23d33c-4709-4320-8359-dea6305b5869-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.325104 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.324840 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7e23d33c-4709-4320-8359-dea6305b5869-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.326692 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.326658 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7e23d33c-4709-4320-8359-dea6305b5869-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.327298 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.327026 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7e23d33c-4709-4320-8359-dea6305b5869-config-out\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.327298 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.327175 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7e23d33c-4709-4320-8359-dea6305b5869-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.327298 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.327270 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7e23d33c-4709-4320-8359-dea6305b5869-web-config\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.327298 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.327270 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7e23d33c-4709-4320-8359-dea6305b5869-config-volume\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.327482 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.327273 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7e23d33c-4709-4320-8359-dea6305b5869-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.327482 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.327359 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e23d33c-4709-4320-8359-dea6305b5869-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.327830 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.327810 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7e23d33c-4709-4320-8359-dea6305b5869-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.328133 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.328115 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e23d33c-4709-4320-8359-dea6305b5869-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.333429 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.333400 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddz4h\" (UniqueName: \"kubernetes.io/projected/7e23d33c-4709-4320-8359-dea6305b5869-kube-api-access-ddz4h\") pod \"alertmanager-main-0\" (UID: \"7e23d33c-4709-4320-8359-dea6305b5869\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.497706 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.497675 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 19:08:11.628316 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:11.628284 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 19:08:11.631504 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:08:11.631474 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e23d33c_4709_4320_8359_dea6305b5869.slice/crio-f847d3fd4c214117b2f57f77c918f09d943278fcfdd3a543794eb20bcdf5f24b WatchSource:0}: Error finding container f847d3fd4c214117b2f57f77c918f09d943278fcfdd3a543794eb20bcdf5f24b: Status 404 returned error can't find the container with id f847d3fd4c214117b2f57f77c918f09d943278fcfdd3a543794eb20bcdf5f24b Jul 08 19:08:12.131480 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:12.131449 2560 generic.go:356] "Generic (PLEG): container finished" podID="7e23d33c-4709-4320-8359-dea6305b5869" containerID="fbbcd392a28a803e2841d983a1a1c62f45349c5fd8db7667bbc77c46b807ee75" exitCode=0 Jul 08 19:08:12.131670 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:12.131558 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7e23d33c-4709-4320-8359-dea6305b5869","Type":"ContainerDied","Data":"fbbcd392a28a803e2841d983a1a1c62f45349c5fd8db7667bbc77c46b807ee75"} Jul 08 19:08:12.131670 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:12.131644 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7e23d33c-4709-4320-8359-dea6305b5869","Type":"ContainerStarted","Data":"f847d3fd4c214117b2f57f77c918f09d943278fcfdd3a543794eb20bcdf5f24b"} Jul 08 19:08:12.355968 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:12.355941 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d215f4f-ea16-449d-a0f7-ba0a0a1f7032" path="/var/lib/kubelet/pods/3d215f4f-ea16-449d-a0f7-ba0a0a1f7032/volumes" Jul 08 19:08:13.139096 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.139067 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7e23d33c-4709-4320-8359-dea6305b5869","Type":"ContainerStarted","Data":"48126b4899d22a55313d1fed7896278ed8c17da50d66f76d6f7da5b1d7c0b4a1"} Jul 08 19:08:13.139435 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.139125 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7e23d33c-4709-4320-8359-dea6305b5869","Type":"ContainerStarted","Data":"a72b84d67d4a4ab368b5cc093a9e523807de79d963abbf0938717741089380f2"} Jul 08 19:08:13.139435 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.139153 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7e23d33c-4709-4320-8359-dea6305b5869","Type":"ContainerStarted","Data":"d1594d453e44b1aa8c241a5cf9c91f5818f61f6b01a79177acf0af4d72bc81f2"} Jul 08 19:08:13.139435 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.139192 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7e23d33c-4709-4320-8359-dea6305b5869","Type":"ContainerStarted","Data":"86191fe6fe025fc6ac198ad44caeba953ab3c4ffee992aa17fab4e47d317051e"} Jul 08 19:08:13.139435 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.139222 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7e23d33c-4709-4320-8359-dea6305b5869","Type":"ContainerStarted","Data":"a16c5791232333fa9fa625c38aef926cdca2ed528c5803c5f77b57afb4206a75"} Jul 08 19:08:13.139435 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.139242 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7e23d33c-4709-4320-8359-dea6305b5869","Type":"ContainerStarted","Data":"e141cce3c42e0025949df7871a4d2c5ab5e8003a91d4ff0f6014d24595cd10a0"} Jul 08 19:08:13.167728 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.167679 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.150947579 podStartE2EDuration="2.167665142s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.717563ms" isStatefulPod=false podCreationTimestamp="2026-07-08 19:08:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:08:13.165312683 +0000 UTC m=+167.343446688" watchObservedRunningTime="2026-07-08 19:08:13.167665142 +0000 UTC m=+167.345799150" Jul 08 19:08:13.312604 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.312567 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-68c889fb65-ns8jm"] Jul 08 19:08:13.315144 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.315122 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.317952 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.317929 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 08 19:08:13.318061 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.317927 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-p7g8t\"" Jul 08 19:08:13.318061 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.317987 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 08 19:08:13.318163 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.318088 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 08 19:08:13.318163 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.318130 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 08 19:08:13.318413 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.318399 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 08 19:08:13.328574 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.328519 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 08 19:08:13.330893 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.330862 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-68c889fb65-ns8jm"] Jul 08 19:08:13.340154 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.340131 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-telemeter-trusted-ca-bundle\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.340256 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.340170 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-telemeter-client-tls\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.340256 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.340204 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-secret-telemeter-client\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.340256 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.340243 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-federate-client-tls\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.340349 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.340295 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-metrics-client-ca\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.340349 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.340318 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lthm\" (UniqueName: \"kubernetes.io/projected/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-kube-api-access-5lthm\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.340349 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.340335 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-serving-certs-ca-bundle\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.340459 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.340363 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.441821 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.441724 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-federate-client-tls\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.441821 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.441812 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-metrics-client-ca\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.442048 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.441845 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lthm\" (UniqueName: \"kubernetes.io/projected/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-kube-api-access-5lthm\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.442048 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.441876 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-serving-certs-ca-bundle\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.442048 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.442013 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.442187 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.442078 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-telemeter-trusted-ca-bundle\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.442187 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.442135 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-telemeter-client-tls\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.442187 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.442172 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-secret-telemeter-client\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.442717 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.442690 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-metrics-client-ca\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.442964 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.442942 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-serving-certs-ca-bundle\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.443172 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.443145 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-telemeter-trusted-ca-bundle\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.445421 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.445292 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-telemeter-client-tls\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.445421 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.445380 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-federate-client-tls\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.446001 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.445959 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.446345 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.446170 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-secret-telemeter-client\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.451588 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.451564 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lthm\" (UniqueName: \"kubernetes.io/projected/927d1f81-c5c6-4639-a1fa-0be3e6d58c07-kube-api-access-5lthm\") pod \"telemeter-client-68c889fb65-ns8jm\" (UID: \"927d1f81-c5c6-4639-a1fa-0be3e6d58c07\") " pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.625414 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.625380 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" Jul 08 19:08:13.780473 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:13.780447 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-68c889fb65-ns8jm"] Jul 08 19:08:13.782401 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:08:13.782371 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod927d1f81_c5c6_4639_a1fa_0be3e6d58c07.slice/crio-9a35d62060e59ae79441dda5d623be3a0d1e3284c8b73d91287cb9963127f58f WatchSource:0}: Error finding container 9a35d62060e59ae79441dda5d623be3a0d1e3284c8b73d91287cb9963127f58f: Status 404 returned error can't find the container with id 9a35d62060e59ae79441dda5d623be3a0d1e3284c8b73d91287cb9963127f58f Jul 08 19:08:14.144822 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:14.144790 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" event={"ID":"927d1f81-c5c6-4639-a1fa-0be3e6d58c07","Type":"ContainerStarted","Data":"9a35d62060e59ae79441dda5d623be3a0d1e3284c8b73d91287cb9963127f58f"} Jul 08 19:08:17.159105 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:17.159063 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" event={"ID":"927d1f81-c5c6-4639-a1fa-0be3e6d58c07","Type":"ContainerStarted","Data":"336cf2ea4b765a013885f4146dbb7396486efaa2b2639d7c61a57c1fd516432f"} Jul 08 19:08:17.159480 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:17.159136 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" event={"ID":"927d1f81-c5c6-4639-a1fa-0be3e6d58c07","Type":"ContainerStarted","Data":"b1b3049b31e1859d68684678e007b0d6bb941e1c2eaacf8f01eaca64e08e5aa7"} Jul 08 19:08:17.159480 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:17.159167 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" event={"ID":"927d1f81-c5c6-4639-a1fa-0be3e6d58c07","Type":"ContainerStarted","Data":"3e000122401c11f223633b3f59b6685ab127b841923b003467c259b7448c69ec"} Jul 08 19:08:17.190051 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:17.190002 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-68c889fb65-ns8jm" podStartSLOduration=1.7172741280000001 podStartE2EDuration="4.189987035s" totalImagesPullingTime="2.472712907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:08:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:08:17.187334549 +0000 UTC m=+171.365468555" watchObservedRunningTime="2026-07-08 19:08:17.189987035 +0000 UTC m=+171.368121042" Jul 08 19:08:17.887921 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:17.887886 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74f447cf9c-2bnqb"] Jul 08 19:08:17.890292 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:17.890276 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:17.903823 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:17.903795 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74f447cf9c-2bnqb"] Jul 08 19:08:17.984059 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:17.984029 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-trusted-ca-bundle\") pod \"console-74f447cf9c-2bnqb\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:17.984059 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:17.984065 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nzxj\" (UniqueName: \"kubernetes.io/projected/909a2102-9a49-493b-a597-3927d36de731-kube-api-access-7nzxj\") pod \"console-74f447cf9c-2bnqb\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:17.984300 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:17.984094 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-oauth-serving-cert\") pod \"console-74f447cf9c-2bnqb\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:17.984300 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:17.984158 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-console-config\") pod \"console-74f447cf9c-2bnqb\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:17.984300 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:17.984190 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-service-ca\") pod \"console-74f447cf9c-2bnqb\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:17.984300 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:17.984216 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/909a2102-9a49-493b-a597-3927d36de731-console-oauth-config\") pod \"console-74f447cf9c-2bnqb\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:17.984300 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:17.984239 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/909a2102-9a49-493b-a597-3927d36de731-console-serving-cert\") pod \"console-74f447cf9c-2bnqb\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:18.085345 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:18.085304 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nzxj\" (UniqueName: \"kubernetes.io/projected/909a2102-9a49-493b-a597-3927d36de731-kube-api-access-7nzxj\") pod \"console-74f447cf9c-2bnqb\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:18.085485 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:18.085364 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-oauth-serving-cert\") pod \"console-74f447cf9c-2bnqb\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:18.085485 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:18.085394 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-console-config\") pod \"console-74f447cf9c-2bnqb\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:18.085485 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:18.085410 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-service-ca\") pod \"console-74f447cf9c-2bnqb\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:18.085485 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:18.085432 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/909a2102-9a49-493b-a597-3927d36de731-console-oauth-config\") pod \"console-74f447cf9c-2bnqb\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:18.085485 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:18.085455 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/909a2102-9a49-493b-a597-3927d36de731-console-serving-cert\") pod \"console-74f447cf9c-2bnqb\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:18.085772 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:18.085605 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-trusted-ca-bundle\") pod \"console-74f447cf9c-2bnqb\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:18.086255 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:18.086227 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-service-ca\") pod \"console-74f447cf9c-2bnqb\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:18.086374 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:18.086234 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-console-config\") pod \"console-74f447cf9c-2bnqb\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:18.086374 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:18.086232 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-oauth-serving-cert\") pod \"console-74f447cf9c-2bnqb\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:18.086374 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:18.086330 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-trusted-ca-bundle\") pod \"console-74f447cf9c-2bnqb\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:18.088181 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:18.088154 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/909a2102-9a49-493b-a597-3927d36de731-console-oauth-config\") pod \"console-74f447cf9c-2bnqb\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:18.088260 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:18.088166 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/909a2102-9a49-493b-a597-3927d36de731-console-serving-cert\") pod \"console-74f447cf9c-2bnqb\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:18.095348 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:18.095316 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nzxj\" (UniqueName: \"kubernetes.io/projected/909a2102-9a49-493b-a597-3927d36de731-kube-api-access-7nzxj\") pod \"console-74f447cf9c-2bnqb\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:18.199323 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:18.199245 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:18.325614 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:18.325590 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74f447cf9c-2bnqb"] Jul 08 19:08:18.328140 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:08:18.328112 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod909a2102_9a49_493b_a597_3927d36de731.slice/crio-b31b00e74ad802f5611dd7d32861b676769c639856a161312cb9fd2b28ac1ac2 WatchSource:0}: Error finding container b31b00e74ad802f5611dd7d32861b676769c639856a161312cb9fd2b28ac1ac2: Status 404 returned error can't find the container with id b31b00e74ad802f5611dd7d32861b676769c639856a161312cb9fd2b28ac1ac2 Jul 08 19:08:19.168960 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:19.168919 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74f447cf9c-2bnqb" event={"ID":"909a2102-9a49-493b-a597-3927d36de731","Type":"ContainerStarted","Data":"1ab795a13266ac363325bbe595e77a8119f3e626b1166f8449fab17e353a1b7f"} Jul 08 19:08:19.169147 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:19.168972 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74f447cf9c-2bnqb" event={"ID":"909a2102-9a49-493b-a597-3927d36de731","Type":"ContainerStarted","Data":"b31b00e74ad802f5611dd7d32861b676769c639856a161312cb9fd2b28ac1ac2"} Jul 08 19:08:19.190253 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:19.190207 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74f447cf9c-2bnqb" podStartSLOduration=2.19018895 podStartE2EDuration="2.19018895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:08:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:08:19.188913415 +0000 UTC m=+173.367047422" watchObservedRunningTime="2026-07-08 19:08:19.19018895 +0000 UTC m=+173.368322958" Jul 08 19:08:28.199650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:28.199606 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:28.200075 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:28.199712 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:28.204599 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:28.204579 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:29.210897 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:29.210864 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:08:29.262682 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:29.262650 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67875678f7-lhrpb"] Jul 08 19:08:54.283709 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.283645 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-67875678f7-lhrpb" podUID="911f8ba6-0c9e-439d-b6d9-2bd635638042" containerName="console" containerID="cri-o://38fc5a102b35c8a1f5a146db867ecf403e64e6c45dc5fae79f25443669e25718" gracePeriod=15 Jul 08 19:08:54.532268 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.532240 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:08:54.592181 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.592111 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/911f8ba6-0c9e-439d-b6d9-2bd635638042-console-serving-cert\") pod \"911f8ba6-0c9e-439d-b6d9-2bd635638042\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " Jul 08 19:08:54.592181 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.592166 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-oauth-serving-cert\") pod \"911f8ba6-0c9e-439d-b6d9-2bd635638042\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " Jul 08 19:08:54.592366 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.592196 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/911f8ba6-0c9e-439d-b6d9-2bd635638042-console-oauth-config\") pod \"911f8ba6-0c9e-439d-b6d9-2bd635638042\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " Jul 08 19:08:54.592366 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.592216 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-service-ca\") pod \"911f8ba6-0c9e-439d-b6d9-2bd635638042\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " Jul 08 19:08:54.592633 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.592586 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "911f8ba6-0c9e-439d-b6d9-2bd635638042" (UID: "911f8ba6-0c9e-439d-b6d9-2bd635638042"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:08:54.592764 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.592667 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-service-ca" (OuterVolumeSpecName: "service-ca") pod "911f8ba6-0c9e-439d-b6d9-2bd635638042" (UID: "911f8ba6-0c9e-439d-b6d9-2bd635638042"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:08:54.594335 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.594314 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/911f8ba6-0c9e-439d-b6d9-2bd635638042-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "911f8ba6-0c9e-439d-b6d9-2bd635638042" (UID: "911f8ba6-0c9e-439d-b6d9-2bd635638042"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:08:54.594426 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.594366 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/911f8ba6-0c9e-439d-b6d9-2bd635638042-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "911f8ba6-0c9e-439d-b6d9-2bd635638042" (UID: "911f8ba6-0c9e-439d-b6d9-2bd635638042"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:08:54.693078 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.693041 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-trusted-ca-bundle\") pod \"911f8ba6-0c9e-439d-b6d9-2bd635638042\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " Jul 08 19:08:54.693078 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.693080 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-console-config\") pod \"911f8ba6-0c9e-439d-b6d9-2bd635638042\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " Jul 08 19:08:54.693320 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.693105 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhxdm\" (UniqueName: \"kubernetes.io/projected/911f8ba6-0c9e-439d-b6d9-2bd635638042-kube-api-access-qhxdm\") pod \"911f8ba6-0c9e-439d-b6d9-2bd635638042\" (UID: \"911f8ba6-0c9e-439d-b6d9-2bd635638042\") " Jul 08 19:08:54.693320 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.693248 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/911f8ba6-0c9e-439d-b6d9-2bd635638042-console-serving-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:08:54.693320 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.693257 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-oauth-serving-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:08:54.693320 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.693267 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/911f8ba6-0c9e-439d-b6d9-2bd635638042-console-oauth-config\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:08:54.693320 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.693276 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-service-ca\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:08:54.693558 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.693372 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "911f8ba6-0c9e-439d-b6d9-2bd635638042" (UID: "911f8ba6-0c9e-439d-b6d9-2bd635638042"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:08:54.693558 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.693396 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-console-config" (OuterVolumeSpecName: "console-config") pod "911f8ba6-0c9e-439d-b6d9-2bd635638042" (UID: "911f8ba6-0c9e-439d-b6d9-2bd635638042"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:08:54.695202 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.695183 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/911f8ba6-0c9e-439d-b6d9-2bd635638042-kube-api-access-qhxdm" (OuterVolumeSpecName: "kube-api-access-qhxdm") pod "911f8ba6-0c9e-439d-b6d9-2bd635638042" (UID: "911f8ba6-0c9e-439d-b6d9-2bd635638042"). InnerVolumeSpecName "kube-api-access-qhxdm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:08:54.794039 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.793989 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-trusted-ca-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:08:54.794039 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.794033 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/911f8ba6-0c9e-439d-b6d9-2bd635638042-console-config\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:08:54.794039 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:54.794046 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhxdm\" (UniqueName: \"kubernetes.io/projected/911f8ba6-0c9e-439d-b6d9-2bd635638042-kube-api-access-qhxdm\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:08:55.303138 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:55.303108 2560 generic.go:356] "Generic (PLEG): container finished" podID="911f8ba6-0c9e-439d-b6d9-2bd635638042" containerID="38fc5a102b35c8a1f5a146db867ecf403e64e6c45dc5fae79f25443669e25718" exitCode=0 Jul 08 19:08:55.303454 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:55.303158 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67875678f7-lhrpb" Jul 08 19:08:55.303454 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:55.303180 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67875678f7-lhrpb" event={"ID":"911f8ba6-0c9e-439d-b6d9-2bd635638042","Type":"ContainerDied","Data":"38fc5a102b35c8a1f5a146db867ecf403e64e6c45dc5fae79f25443669e25718"} Jul 08 19:08:55.303454 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:55.303221 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67875678f7-lhrpb" event={"ID":"911f8ba6-0c9e-439d-b6d9-2bd635638042","Type":"ContainerDied","Data":"578efcfee4f88a081d8dba21cacce893c53fe4e6c99c6a5770ea6958fbe886f6"} Jul 08 19:08:55.303454 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:55.303240 2560 scope.go:117] "RemoveContainer" containerID="38fc5a102b35c8a1f5a146db867ecf403e64e6c45dc5fae79f25443669e25718" Jul 08 19:08:55.311358 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:55.311342 2560 scope.go:117] "RemoveContainer" containerID="38fc5a102b35c8a1f5a146db867ecf403e64e6c45dc5fae79f25443669e25718" Jul 08 19:08:55.311619 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:08:55.311602 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38fc5a102b35c8a1f5a146db867ecf403e64e6c45dc5fae79f25443669e25718\": container with ID starting with 38fc5a102b35c8a1f5a146db867ecf403e64e6c45dc5fae79f25443669e25718 not found: ID does not exist" containerID="38fc5a102b35c8a1f5a146db867ecf403e64e6c45dc5fae79f25443669e25718" Jul 08 19:08:55.311691 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:55.311631 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38fc5a102b35c8a1f5a146db867ecf403e64e6c45dc5fae79f25443669e25718"} err="failed to get container status \"38fc5a102b35c8a1f5a146db867ecf403e64e6c45dc5fae79f25443669e25718\": rpc error: code = NotFound desc = could not find container \"38fc5a102b35c8a1f5a146db867ecf403e64e6c45dc5fae79f25443669e25718\": container with ID starting with 38fc5a102b35c8a1f5a146db867ecf403e64e6c45dc5fae79f25443669e25718 not found: ID does not exist" Jul 08 19:08:55.323657 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:55.323632 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67875678f7-lhrpb"] Jul 08 19:08:55.327598 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:55.327577 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67875678f7-lhrpb"] Jul 08 19:08:56.353910 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:08:56.353879 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="911f8ba6-0c9e-439d-b6d9-2bd635638042" path="/var/lib/kubelet/pods/911f8ba6-0c9e-439d-b6d9-2bd635638042/volumes" Jul 08 19:09:33.880852 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:33.880820 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d596bd57c-mtj8c"] Jul 08 19:09:33.881267 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:33.881131 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="911f8ba6-0c9e-439d-b6d9-2bd635638042" containerName="console" Jul 08 19:09:33.881267 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:33.881143 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="911f8ba6-0c9e-439d-b6d9-2bd635638042" containerName="console" Jul 08 19:09:33.881267 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:33.881184 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="911f8ba6-0c9e-439d-b6d9-2bd635638042" containerName="console" Jul 08 19:09:33.885233 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:33.885217 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:33.938368 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:33.938343 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d596bd57c-mtj8c"] Jul 08 19:09:33.991562 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:33.991512 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-service-ca\") pod \"console-d596bd57c-mtj8c\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:33.991562 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:33.991563 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-console-config\") pod \"console-d596bd57c-mtj8c\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:33.991749 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:33.991598 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-oauth-serving-cert\") pod \"console-d596bd57c-mtj8c\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:33.991749 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:33.991618 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-trusted-ca-bundle\") pod \"console-d596bd57c-mtj8c\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:33.991749 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:33.991639 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j6s4\" (UniqueName: \"kubernetes.io/projected/cb1c7709-e00a-45fb-b740-48fbba056675-kube-api-access-9j6s4\") pod \"console-d596bd57c-mtj8c\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:33.991749 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:33.991706 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb1c7709-e00a-45fb-b740-48fbba056675-console-oauth-config\") pod \"console-d596bd57c-mtj8c\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:33.991749 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:33.991733 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb1c7709-e00a-45fb-b740-48fbba056675-console-serving-cert\") pod \"console-d596bd57c-mtj8c\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:34.092607 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:34.092576 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-service-ca\") pod \"console-d596bd57c-mtj8c\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:34.092607 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:34.092609 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-console-config\") pod \"console-d596bd57c-mtj8c\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:34.092808 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:34.092657 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-oauth-serving-cert\") pod \"console-d596bd57c-mtj8c\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:34.092808 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:34.092686 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-trusted-ca-bundle\") pod \"console-d596bd57c-mtj8c\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:34.092808 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:34.092717 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j6s4\" (UniqueName: \"kubernetes.io/projected/cb1c7709-e00a-45fb-b740-48fbba056675-kube-api-access-9j6s4\") pod \"console-d596bd57c-mtj8c\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:34.092808 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:34.092761 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb1c7709-e00a-45fb-b740-48fbba056675-console-oauth-config\") pod \"console-d596bd57c-mtj8c\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:34.092808 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:34.092786 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb1c7709-e00a-45fb-b740-48fbba056675-console-serving-cert\") pod \"console-d596bd57c-mtj8c\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:34.093402 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:34.093374 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-service-ca\") pod \"console-d596bd57c-mtj8c\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:34.093553 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:34.093423 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-oauth-serving-cert\") pod \"console-d596bd57c-mtj8c\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:34.093553 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:34.093499 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-console-config\") pod \"console-d596bd57c-mtj8c\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:34.093661 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:34.093562 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-trusted-ca-bundle\") pod \"console-d596bd57c-mtj8c\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:34.095845 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:34.095821 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb1c7709-e00a-45fb-b740-48fbba056675-console-oauth-config\") pod \"console-d596bd57c-mtj8c\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:34.095938 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:34.095883 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb1c7709-e00a-45fb-b740-48fbba056675-console-serving-cert\") pod \"console-d596bd57c-mtj8c\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:34.101391 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:34.101358 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j6s4\" (UniqueName: \"kubernetes.io/projected/cb1c7709-e00a-45fb-b740-48fbba056675-kube-api-access-9j6s4\") pod \"console-d596bd57c-mtj8c\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:34.193432 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:34.193405 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:34.327708 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:34.327682 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d596bd57c-mtj8c"] Jul 08 19:09:34.329322 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:09:34.329292 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb1c7709_e00a_45fb_b740_48fbba056675.slice/crio-ca914be693b6f1eafd2d503c50c00c3d2c04bc076e88b9e9283c90937301cf2b WatchSource:0}: Error finding container ca914be693b6f1eafd2d503c50c00c3d2c04bc076e88b9e9283c90937301cf2b: Status 404 returned error can't find the container with id ca914be693b6f1eafd2d503c50c00c3d2c04bc076e88b9e9283c90937301cf2b Jul 08 19:09:34.448091 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:34.448013 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d596bd57c-mtj8c" event={"ID":"cb1c7709-e00a-45fb-b740-48fbba056675","Type":"ContainerStarted","Data":"611605f551057392ff3af1fa9ac1497cc6829031528dc7e42a05f80dab5aa0d5"} Jul 08 19:09:34.448235 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:34.448093 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d596bd57c-mtj8c" event={"ID":"cb1c7709-e00a-45fb-b740-48fbba056675","Type":"ContainerStarted","Data":"ca914be693b6f1eafd2d503c50c00c3d2c04bc076e88b9e9283c90937301cf2b"} Jul 08 19:09:34.471568 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:34.471499 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d596bd57c-mtj8c" podStartSLOduration=1.471484529 podStartE2EDuration="1.471484529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:09:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:09:34.469991895 +0000 UTC m=+248.648125914" watchObservedRunningTime="2026-07-08 19:09:34.471484529 +0000 UTC m=+248.649618535" Jul 08 19:09:44.194003 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:44.193965 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:44.194489 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:44.194026 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:44.198724 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:44.198701 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:44.487699 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:44.487620 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:09:44.538595 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:09:44.538558 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74f447cf9c-2bnqb"] Jul 08 19:10:09.559491 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.559451 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-74f447cf9c-2bnqb" podUID="909a2102-9a49-493b-a597-3927d36de731" containerName="console" containerID="cri-o://1ab795a13266ac363325bbe595e77a8119f3e626b1166f8449fab17e353a1b7f" gracePeriod=15 Jul 08 19:10:09.809250 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.809220 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:10:09.861108 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.861054 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-trusted-ca-bundle\") pod \"909a2102-9a49-493b-a597-3927d36de731\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " Jul 08 19:10:09.861108 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.861106 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/909a2102-9a49-493b-a597-3927d36de731-console-oauth-config\") pod \"909a2102-9a49-493b-a597-3927d36de731\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " Jul 08 19:10:09.861264 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.861124 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-oauth-serving-cert\") pod \"909a2102-9a49-493b-a597-3927d36de731\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " Jul 08 19:10:09.861264 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.861150 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nzxj\" (UniqueName: \"kubernetes.io/projected/909a2102-9a49-493b-a597-3927d36de731-kube-api-access-7nzxj\") pod \"909a2102-9a49-493b-a597-3927d36de731\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " Jul 08 19:10:09.861264 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.861171 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-service-ca\") pod \"909a2102-9a49-493b-a597-3927d36de731\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " Jul 08 19:10:09.861264 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.861226 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/909a2102-9a49-493b-a597-3927d36de731-console-serving-cert\") pod \"909a2102-9a49-493b-a597-3927d36de731\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " Jul 08 19:10:09.861264 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.861257 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-console-config\") pod \"909a2102-9a49-493b-a597-3927d36de731\" (UID: \"909a2102-9a49-493b-a597-3927d36de731\") " Jul 08 19:10:09.861616 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.861550 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "909a2102-9a49-493b-a597-3927d36de731" (UID: "909a2102-9a49-493b-a597-3927d36de731"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:10:09.861722 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.861642 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-service-ca" (OuterVolumeSpecName: "service-ca") pod "909a2102-9a49-493b-a597-3927d36de731" (UID: "909a2102-9a49-493b-a597-3927d36de731"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:10:09.861722 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.861679 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "909a2102-9a49-493b-a597-3927d36de731" (UID: "909a2102-9a49-493b-a597-3927d36de731"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:10:09.861832 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.861740 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-console-config" (OuterVolumeSpecName: "console-config") pod "909a2102-9a49-493b-a597-3927d36de731" (UID: "909a2102-9a49-493b-a597-3927d36de731"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:10:09.863232 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.863202 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/909a2102-9a49-493b-a597-3927d36de731-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "909a2102-9a49-493b-a597-3927d36de731" (UID: "909a2102-9a49-493b-a597-3927d36de731"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:10:09.863423 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.863401 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/909a2102-9a49-493b-a597-3927d36de731-kube-api-access-7nzxj" (OuterVolumeSpecName: "kube-api-access-7nzxj") pod "909a2102-9a49-493b-a597-3927d36de731" (UID: "909a2102-9a49-493b-a597-3927d36de731"). InnerVolumeSpecName "kube-api-access-7nzxj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:10:09.863503 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.863454 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/909a2102-9a49-493b-a597-3927d36de731-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "909a2102-9a49-493b-a597-3927d36de731" (UID: "909a2102-9a49-493b-a597-3927d36de731"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:10:09.961960 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.961930 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/909a2102-9a49-493b-a597-3927d36de731-console-oauth-config\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:10:09.961960 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.961957 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-oauth-serving-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:10:09.962096 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.961971 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nzxj\" (UniqueName: \"kubernetes.io/projected/909a2102-9a49-493b-a597-3927d36de731-kube-api-access-7nzxj\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:10:09.962096 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.961985 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-service-ca\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:10:09.962096 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.961997 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/909a2102-9a49-493b-a597-3927d36de731-console-serving-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:10:09.962096 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.962008 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-console-config\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:10:09.962096 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:09.962020 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/909a2102-9a49-493b-a597-3927d36de731-trusted-ca-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:10:10.583032 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:10.583004 2560 generic.go:356] "Generic (PLEG): container finished" podID="909a2102-9a49-493b-a597-3927d36de731" containerID="1ab795a13266ac363325bbe595e77a8119f3e626b1166f8449fab17e353a1b7f" exitCode=0 Jul 08 19:10:10.583372 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:10.583046 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74f447cf9c-2bnqb" Jul 08 19:10:10.583372 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:10.583089 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74f447cf9c-2bnqb" event={"ID":"909a2102-9a49-493b-a597-3927d36de731","Type":"ContainerDied","Data":"1ab795a13266ac363325bbe595e77a8119f3e626b1166f8449fab17e353a1b7f"} Jul 08 19:10:10.583372 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:10.583136 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74f447cf9c-2bnqb" event={"ID":"909a2102-9a49-493b-a597-3927d36de731","Type":"ContainerDied","Data":"b31b00e74ad802f5611dd7d32861b676769c639856a161312cb9fd2b28ac1ac2"} Jul 08 19:10:10.583372 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:10.583155 2560 scope.go:117] "RemoveContainer" containerID="1ab795a13266ac363325bbe595e77a8119f3e626b1166f8449fab17e353a1b7f" Jul 08 19:10:10.590972 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:10.590958 2560 scope.go:117] "RemoveContainer" containerID="1ab795a13266ac363325bbe595e77a8119f3e626b1166f8449fab17e353a1b7f" Jul 08 19:10:10.591211 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:10:10.591191 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ab795a13266ac363325bbe595e77a8119f3e626b1166f8449fab17e353a1b7f\": container with ID starting with 1ab795a13266ac363325bbe595e77a8119f3e626b1166f8449fab17e353a1b7f not found: ID does not exist" containerID="1ab795a13266ac363325bbe595e77a8119f3e626b1166f8449fab17e353a1b7f" Jul 08 19:10:10.591268 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:10.591221 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ab795a13266ac363325bbe595e77a8119f3e626b1166f8449fab17e353a1b7f"} err="failed to get container status \"1ab795a13266ac363325bbe595e77a8119f3e626b1166f8449fab17e353a1b7f\": rpc error: code = NotFound desc = could not find container \"1ab795a13266ac363325bbe595e77a8119f3e626b1166f8449fab17e353a1b7f\": container with ID starting with 1ab795a13266ac363325bbe595e77a8119f3e626b1166f8449fab17e353a1b7f not found: ID does not exist" Jul 08 19:10:10.616634 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:10.616582 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74f447cf9c-2bnqb"] Jul 08 19:10:10.619586 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:10.619568 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-74f447cf9c-2bnqb"] Jul 08 19:10:12.353599 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:12.353566 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="909a2102-9a49-493b-a597-3927d36de731" path="/var/lib/kubelet/pods/909a2102-9a49-493b-a597-3927d36de731/volumes" Jul 08 19:10:26.181017 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:26.180991 2560 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 19:10:31.921022 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:31.920987 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm"] Jul 08 19:10:31.922992 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:31.921267 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="909a2102-9a49-493b-a597-3927d36de731" containerName="console" Jul 08 19:10:31.922992 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:31.921277 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="909a2102-9a49-493b-a597-3927d36de731" containerName="console" Jul 08 19:10:31.922992 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:31.921341 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="909a2102-9a49-493b-a597-3927d36de731" containerName="console" Jul 08 19:10:31.924200 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:31.924186 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm" Jul 08 19:10:31.928334 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:31.928304 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 19:10:31.929247 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:31.929226 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 19:10:31.930567 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:31.930220 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-lmpd8\"" Jul 08 19:10:31.939287 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:31.939264 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm"] Jul 08 19:10:32.011842 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:32.011815 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/44b19a24-9736-4ef9-b0ff-b22114479e58-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm\" (UID: \"44b19a24-9736-4ef9-b0ff-b22114479e58\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm" Jul 08 19:10:32.011946 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:32.011851 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjqp7\" (UniqueName: \"kubernetes.io/projected/44b19a24-9736-4ef9-b0ff-b22114479e58-kube-api-access-xjqp7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm\" (UID: \"44b19a24-9736-4ef9-b0ff-b22114479e58\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm" Jul 08 19:10:32.011946 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:32.011881 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/44b19a24-9736-4ef9-b0ff-b22114479e58-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm\" (UID: \"44b19a24-9736-4ef9-b0ff-b22114479e58\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm" Jul 08 19:10:32.112871 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:32.112850 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/44b19a24-9736-4ef9-b0ff-b22114479e58-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm\" (UID: \"44b19a24-9736-4ef9-b0ff-b22114479e58\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm" Jul 08 19:10:32.112931 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:32.112897 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/44b19a24-9736-4ef9-b0ff-b22114479e58-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm\" (UID: \"44b19a24-9736-4ef9-b0ff-b22114479e58\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm" Jul 08 19:10:32.112969 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:32.112930 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjqp7\" (UniqueName: \"kubernetes.io/projected/44b19a24-9736-4ef9-b0ff-b22114479e58-kube-api-access-xjqp7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm\" (UID: \"44b19a24-9736-4ef9-b0ff-b22114479e58\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm" Jul 08 19:10:32.113192 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:32.113177 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/44b19a24-9736-4ef9-b0ff-b22114479e58-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm\" (UID: \"44b19a24-9736-4ef9-b0ff-b22114479e58\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm" Jul 08 19:10:32.113263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:32.113246 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/44b19a24-9736-4ef9-b0ff-b22114479e58-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm\" (UID: \"44b19a24-9736-4ef9-b0ff-b22114479e58\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm" Jul 08 19:10:32.122415 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:32.122397 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjqp7\" (UniqueName: \"kubernetes.io/projected/44b19a24-9736-4ef9-b0ff-b22114479e58-kube-api-access-xjqp7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm\" (UID: \"44b19a24-9736-4ef9-b0ff-b22114479e58\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm" Jul 08 19:10:32.232667 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:32.232616 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm" Jul 08 19:10:32.361000 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:32.360963 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm"] Jul 08 19:10:32.365195 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:10:32.365172 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44b19a24_9736_4ef9_b0ff_b22114479e58.slice/crio-c2de3107e227f71c49f8b74e696f48243e0c38614e0f43b3834873bbd3cd7061 WatchSource:0}: Error finding container c2de3107e227f71c49f8b74e696f48243e0c38614e0f43b3834873bbd3cd7061: Status 404 returned error can't find the container with id c2de3107e227f71c49f8b74e696f48243e0c38614e0f43b3834873bbd3cd7061 Jul 08 19:10:32.367118 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:32.367101 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:10:32.664470 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:32.664441 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm" event={"ID":"44b19a24-9736-4ef9-b0ff-b22114479e58","Type":"ContainerStarted","Data":"c2de3107e227f71c49f8b74e696f48243e0c38614e0f43b3834873bbd3cd7061"} Jul 08 19:10:41.702210 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:41.702180 2560 generic.go:356] "Generic (PLEG): container finished" podID="44b19a24-9736-4ef9-b0ff-b22114479e58" containerID="ff8927c8adc00cb16cf33f4bfa987bac2ff4ed3ed748df292c468ed1d37dbd29" exitCode=0 Jul 08 19:10:41.702704 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:41.702228 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm" event={"ID":"44b19a24-9736-4ef9-b0ff-b22114479e58","Type":"ContainerDied","Data":"ff8927c8adc00cb16cf33f4bfa987bac2ff4ed3ed748df292c468ed1d37dbd29"} Jul 08 19:10:43.712380 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:43.712344 2560 generic.go:356] "Generic (PLEG): container finished" podID="44b19a24-9736-4ef9-b0ff-b22114479e58" containerID="84f6e5abd12fb95e0571f8f34a667524684b52308e32a79c72ea47a5a1dc8d53" exitCode=0 Jul 08 19:10:43.712798 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:43.712433 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm" event={"ID":"44b19a24-9736-4ef9-b0ff-b22114479e58","Type":"ContainerDied","Data":"84f6e5abd12fb95e0571f8f34a667524684b52308e32a79c72ea47a5a1dc8d53"} Jul 08 19:10:54.759507 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:54.759469 2560 generic.go:356] "Generic (PLEG): container finished" podID="44b19a24-9736-4ef9-b0ff-b22114479e58" containerID="3f564bfd7f86bad3449da2d60fdccf5bf2baeaa2d8187c503151e7967a17012f" exitCode=0 Jul 08 19:10:54.759915 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:54.759560 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm" event={"ID":"44b19a24-9736-4ef9-b0ff-b22114479e58","Type":"ContainerDied","Data":"3f564bfd7f86bad3449da2d60fdccf5bf2baeaa2d8187c503151e7967a17012f"} Jul 08 19:10:55.914815 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:55.914786 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm" Jul 08 19:10:56.015789 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:56.015694 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjqp7\" (UniqueName: \"kubernetes.io/projected/44b19a24-9736-4ef9-b0ff-b22114479e58-kube-api-access-xjqp7\") pod \"44b19a24-9736-4ef9-b0ff-b22114479e58\" (UID: \"44b19a24-9736-4ef9-b0ff-b22114479e58\") " Jul 08 19:10:56.015789 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:56.015765 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/44b19a24-9736-4ef9-b0ff-b22114479e58-util\") pod \"44b19a24-9736-4ef9-b0ff-b22114479e58\" (UID: \"44b19a24-9736-4ef9-b0ff-b22114479e58\") " Jul 08 19:10:56.015979 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:56.015808 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/44b19a24-9736-4ef9-b0ff-b22114479e58-bundle\") pod \"44b19a24-9736-4ef9-b0ff-b22114479e58\" (UID: \"44b19a24-9736-4ef9-b0ff-b22114479e58\") " Jul 08 19:10:56.016553 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:56.016454 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44b19a24-9736-4ef9-b0ff-b22114479e58-bundle" (OuterVolumeSpecName: "bundle") pod "44b19a24-9736-4ef9-b0ff-b22114479e58" (UID: "44b19a24-9736-4ef9-b0ff-b22114479e58"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:10:56.018025 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:56.018002 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44b19a24-9736-4ef9-b0ff-b22114479e58-kube-api-access-xjqp7" (OuterVolumeSpecName: "kube-api-access-xjqp7") pod "44b19a24-9736-4ef9-b0ff-b22114479e58" (UID: "44b19a24-9736-4ef9-b0ff-b22114479e58"). InnerVolumeSpecName "kube-api-access-xjqp7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:10:56.021641 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:56.021616 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44b19a24-9736-4ef9-b0ff-b22114479e58-util" (OuterVolumeSpecName: "util") pod "44b19a24-9736-4ef9-b0ff-b22114479e58" (UID: "44b19a24-9736-4ef9-b0ff-b22114479e58"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:10:56.116847 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:56.116814 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/44b19a24-9736-4ef9-b0ff-b22114479e58-util\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:10:56.116847 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:56.116842 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/44b19a24-9736-4ef9-b0ff-b22114479e58-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:10:56.116847 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:56.116852 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjqp7\" (UniqueName: \"kubernetes.io/projected/44b19a24-9736-4ef9-b0ff-b22114479e58-kube-api-access-xjqp7\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:10:56.768344 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:56.768313 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm" Jul 08 19:10:56.768344 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:56.768320 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540m4rcm" event={"ID":"44b19a24-9736-4ef9-b0ff-b22114479e58","Type":"ContainerDied","Data":"c2de3107e227f71c49f8b74e696f48243e0c38614e0f43b3834873bbd3cd7061"} Jul 08 19:10:56.768664 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:56.768367 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c2de3107e227f71c49f8b74e696f48243e0c38614e0f43b3834873bbd3cd7061" Jul 08 19:10:59.765121 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:59.765088 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4trqs"] Jul 08 19:10:59.765495 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:59.765380 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44b19a24-9736-4ef9-b0ff-b22114479e58" containerName="util" Jul 08 19:10:59.765495 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:59.765390 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="44b19a24-9736-4ef9-b0ff-b22114479e58" containerName="util" Jul 08 19:10:59.765495 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:59.765403 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44b19a24-9736-4ef9-b0ff-b22114479e58" containerName="pull" Jul 08 19:10:59.765495 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:59.765408 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="44b19a24-9736-4ef9-b0ff-b22114479e58" containerName="pull" Jul 08 19:10:59.765495 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:59.765415 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44b19a24-9736-4ef9-b0ff-b22114479e58" containerName="extract" Jul 08 19:10:59.765495 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:59.765421 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="44b19a24-9736-4ef9-b0ff-b22114479e58" containerName="extract" Jul 08 19:10:59.765495 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:59.765467 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="44b19a24-9736-4ef9-b0ff-b22114479e58" containerName="extract" Jul 08 19:10:59.767603 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:59.767588 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4trqs" Jul 08 19:10:59.770254 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:59.770233 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-t9r9v\"" Jul 08 19:10:59.770414 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:59.770393 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 08 19:10:59.770740 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:59.770728 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:10:59.786341 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:59.786319 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4trqs"] Jul 08 19:10:59.947988 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:59.947949 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s8wm\" (UniqueName: \"kubernetes.io/projected/2c409bbd-6815-424a-8044-b657365f230b-kube-api-access-6s8wm\") pod \"cert-manager-operator-controller-manager-55c444bf86-4trqs\" (UID: \"2c409bbd-6815-424a-8044-b657365f230b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4trqs" Jul 08 19:10:59.948166 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:10:59.948069 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c409bbd-6815-424a-8044-b657365f230b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4trqs\" (UID: \"2c409bbd-6815-424a-8044-b657365f230b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4trqs" Jul 08 19:11:00.049035 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:00.048929 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c409bbd-6815-424a-8044-b657365f230b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4trqs\" (UID: \"2c409bbd-6815-424a-8044-b657365f230b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4trqs" Jul 08 19:11:00.049035 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:00.048985 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s8wm\" (UniqueName: \"kubernetes.io/projected/2c409bbd-6815-424a-8044-b657365f230b-kube-api-access-6s8wm\") pod \"cert-manager-operator-controller-manager-55c444bf86-4trqs\" (UID: \"2c409bbd-6815-424a-8044-b657365f230b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4trqs" Jul 08 19:11:00.049436 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:00.049409 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c409bbd-6815-424a-8044-b657365f230b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-4trqs\" (UID: \"2c409bbd-6815-424a-8044-b657365f230b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4trqs" Jul 08 19:11:00.058331 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:00.058307 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s8wm\" (UniqueName: \"kubernetes.io/projected/2c409bbd-6815-424a-8044-b657365f230b-kube-api-access-6s8wm\") pod \"cert-manager-operator-controller-manager-55c444bf86-4trqs\" (UID: \"2c409bbd-6815-424a-8044-b657365f230b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4trqs" Jul 08 19:11:00.083177 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:00.083150 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4trqs" Jul 08 19:11:00.220427 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:00.220398 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4trqs"] Jul 08 19:11:00.223452 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:11:00.223427 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c409bbd_6815_424a_8044_b657365f230b.slice/crio-c8e4cb3157f55eb7725e0f956197981b2470c5c4d4c7a346fb027de3996c3976 WatchSource:0}: Error finding container c8e4cb3157f55eb7725e0f956197981b2470c5c4d4c7a346fb027de3996c3976: Status 404 returned error can't find the container with id c8e4cb3157f55eb7725e0f956197981b2470c5c4d4c7a346fb027de3996c3976 Jul 08 19:11:00.786132 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:00.786098 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4trqs" event={"ID":"2c409bbd-6815-424a-8044-b657365f230b","Type":"ContainerStarted","Data":"c8e4cb3157f55eb7725e0f956197981b2470c5c4d4c7a346fb027de3996c3976"} Jul 08 19:11:02.801648 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:02.801608 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4trqs" event={"ID":"2c409bbd-6815-424a-8044-b657365f230b","Type":"ContainerStarted","Data":"c5b73b4251c467e4fdd7ebf258cdcfe8e1a3b3a0df22986744a6e29dc7c1994e"} Jul 08 19:11:02.850771 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:02.850709 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-4trqs" podStartSLOduration=2.009102818 podStartE2EDuration="3.85067653s" totalImagesPullingTime="1.841573712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:10:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:02.848688698 +0000 UTC m=+337.026822706" watchObservedRunningTime="2026-07-08 19:11:02.85067653 +0000 UTC m=+337.028810536" Jul 08 19:11:03.181258 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:03.181218 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4mshg"] Jul 08 19:11:03.183811 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:03.183793 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-4mshg" Jul 08 19:11:03.186871 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:03.186853 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-djjhj\"" Jul 08 19:11:03.187283 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:03.187267 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 19:11:03.187359 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:03.187342 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 19:11:03.195467 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:03.195442 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4mshg"] Jul 08 19:11:03.276346 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:03.276305 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ecbabcc6-70d7-4ba3-bde5-b0fad9af54bc-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4mshg\" (UID: \"ecbabcc6-70d7-4ba3-bde5-b0fad9af54bc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4mshg" Jul 08 19:11:03.276346 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:03.276351 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggclw\" (UniqueName: \"kubernetes.io/projected/ecbabcc6-70d7-4ba3-bde5-b0fad9af54bc-kube-api-access-ggclw\") pod \"cert-manager-webhook-6486ffd997-4mshg\" (UID: \"ecbabcc6-70d7-4ba3-bde5-b0fad9af54bc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4mshg" Jul 08 19:11:03.377718 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:03.377676 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ecbabcc6-70d7-4ba3-bde5-b0fad9af54bc-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4mshg\" (UID: \"ecbabcc6-70d7-4ba3-bde5-b0fad9af54bc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4mshg" Jul 08 19:11:03.377718 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:03.377717 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggclw\" (UniqueName: \"kubernetes.io/projected/ecbabcc6-70d7-4ba3-bde5-b0fad9af54bc-kube-api-access-ggclw\") pod \"cert-manager-webhook-6486ffd997-4mshg\" (UID: \"ecbabcc6-70d7-4ba3-bde5-b0fad9af54bc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4mshg" Jul 08 19:11:03.408381 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:03.408344 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ecbabcc6-70d7-4ba3-bde5-b0fad9af54bc-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-4mshg\" (UID: \"ecbabcc6-70d7-4ba3-bde5-b0fad9af54bc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4mshg" Jul 08 19:11:03.409432 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:03.409414 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggclw\" (UniqueName: \"kubernetes.io/projected/ecbabcc6-70d7-4ba3-bde5-b0fad9af54bc-kube-api-access-ggclw\") pod \"cert-manager-webhook-6486ffd997-4mshg\" (UID: \"ecbabcc6-70d7-4ba3-bde5-b0fad9af54bc\") " pod="cert-manager/cert-manager-webhook-6486ffd997-4mshg" Jul 08 19:11:03.506147 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:03.506065 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-4mshg" Jul 08 19:11:03.652080 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:03.652057 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-4mshg"] Jul 08 19:11:03.654494 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:11:03.654471 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podecbabcc6_70d7_4ba3_bde5_b0fad9af54bc.slice/crio-d94f72d26c25cf7d74e5b64d284a5c329a85739cbbef0b65529d4c96e0130dc0 WatchSource:0}: Error finding container d94f72d26c25cf7d74e5b64d284a5c329a85739cbbef0b65529d4c96e0130dc0: Status 404 returned error can't find the container with id d94f72d26c25cf7d74e5b64d284a5c329a85739cbbef0b65529d4c96e0130dc0 Jul 08 19:11:03.806993 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:03.806909 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-4mshg" event={"ID":"ecbabcc6-70d7-4ba3-bde5-b0fad9af54bc","Type":"ContainerStarted","Data":"d94f72d26c25cf7d74e5b64d284a5c329a85739cbbef0b65529d4c96e0130dc0"} Jul 08 19:11:04.414177 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:04.414143 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s"] Jul 08 19:11:04.417593 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:04.417578 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s" Jul 08 19:11:04.420002 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:04.419981 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 19:11:04.420202 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:04.420187 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-lmpd8\"" Jul 08 19:11:04.420963 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:04.420946 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 19:11:04.427660 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:04.427641 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s"] Jul 08 19:11:04.485491 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:04.485453 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d608c8e3-773f-4690-b1c8-ebed82fc79a5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s\" (UID: \"d608c8e3-773f-4690-b1c8-ebed82fc79a5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s" Jul 08 19:11:04.485693 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:04.485512 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d608c8e3-773f-4690-b1c8-ebed82fc79a5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s\" (UID: \"d608c8e3-773f-4690-b1c8-ebed82fc79a5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s" Jul 08 19:11:04.485693 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:04.485619 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5t65\" (UniqueName: \"kubernetes.io/projected/d608c8e3-773f-4690-b1c8-ebed82fc79a5-kube-api-access-c5t65\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s\" (UID: \"d608c8e3-773f-4690-b1c8-ebed82fc79a5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s" Jul 08 19:11:04.586343 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:04.586308 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d608c8e3-773f-4690-b1c8-ebed82fc79a5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s\" (UID: \"d608c8e3-773f-4690-b1c8-ebed82fc79a5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s" Jul 08 19:11:04.586554 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:04.586354 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d608c8e3-773f-4690-b1c8-ebed82fc79a5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s\" (UID: \"d608c8e3-773f-4690-b1c8-ebed82fc79a5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s" Jul 08 19:11:04.586554 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:04.586408 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5t65\" (UniqueName: \"kubernetes.io/projected/d608c8e3-773f-4690-b1c8-ebed82fc79a5-kube-api-access-c5t65\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s\" (UID: \"d608c8e3-773f-4690-b1c8-ebed82fc79a5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s" Jul 08 19:11:04.586832 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:04.586805 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d608c8e3-773f-4690-b1c8-ebed82fc79a5-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s\" (UID: \"d608c8e3-773f-4690-b1c8-ebed82fc79a5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s" Jul 08 19:11:04.586832 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:04.586828 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d608c8e3-773f-4690-b1c8-ebed82fc79a5-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s\" (UID: \"d608c8e3-773f-4690-b1c8-ebed82fc79a5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s" Jul 08 19:11:04.597170 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:04.597144 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5t65\" (UniqueName: \"kubernetes.io/projected/d608c8e3-773f-4690-b1c8-ebed82fc79a5-kube-api-access-c5t65\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s\" (UID: \"d608c8e3-773f-4690-b1c8-ebed82fc79a5\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s" Jul 08 19:11:04.727400 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:04.727318 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s" Jul 08 19:11:04.884430 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:04.884337 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s"] Jul 08 19:11:04.888421 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:11:04.888382 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd608c8e3_773f_4690_b1c8_ebed82fc79a5.slice/crio-2cf8560c9cdb60de0b5bccd80c25c63e110035843d04880ac8218507bd176cd3 WatchSource:0}: Error finding container 2cf8560c9cdb60de0b5bccd80c25c63e110035843d04880ac8218507bd176cd3: Status 404 returned error can't find the container with id 2cf8560c9cdb60de0b5bccd80c25c63e110035843d04880ac8218507bd176cd3 Jul 08 19:11:05.296419 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:05.296338 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8kkhb"] Jul 08 19:11:05.298779 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:05.298755 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8kkhb" Jul 08 19:11:05.301326 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:05.301305 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-kdwhm\"" Jul 08 19:11:05.308001 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:05.307960 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8kkhb"] Jul 08 19:11:05.392995 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:05.392964 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b07d0123-7b08-48c6-9de8-5a6c8be269a0-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8kkhb\" (UID: \"b07d0123-7b08-48c6-9de8-5a6c8be269a0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8kkhb" Jul 08 19:11:05.393176 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:05.393078 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwj28\" (UniqueName: \"kubernetes.io/projected/b07d0123-7b08-48c6-9de8-5a6c8be269a0-kube-api-access-cwj28\") pod \"cert-manager-cainjector-559cdd8c87-8kkhb\" (UID: \"b07d0123-7b08-48c6-9de8-5a6c8be269a0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8kkhb" Jul 08 19:11:05.494148 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:05.494115 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwj28\" (UniqueName: \"kubernetes.io/projected/b07d0123-7b08-48c6-9de8-5a6c8be269a0-kube-api-access-cwj28\") pod \"cert-manager-cainjector-559cdd8c87-8kkhb\" (UID: \"b07d0123-7b08-48c6-9de8-5a6c8be269a0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8kkhb" Jul 08 19:11:05.494326 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:05.494242 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b07d0123-7b08-48c6-9de8-5a6c8be269a0-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8kkhb\" (UID: \"b07d0123-7b08-48c6-9de8-5a6c8be269a0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8kkhb" Jul 08 19:11:05.503023 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:05.502996 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b07d0123-7b08-48c6-9de8-5a6c8be269a0-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8kkhb\" (UID: \"b07d0123-7b08-48c6-9de8-5a6c8be269a0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8kkhb" Jul 08 19:11:05.503243 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:05.503223 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwj28\" (UniqueName: \"kubernetes.io/projected/b07d0123-7b08-48c6-9de8-5a6c8be269a0-kube-api-access-cwj28\") pod \"cert-manager-cainjector-559cdd8c87-8kkhb\" (UID: \"b07d0123-7b08-48c6-9de8-5a6c8be269a0\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8kkhb" Jul 08 19:11:05.611274 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:05.611245 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8kkhb" Jul 08 19:11:05.818522 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:05.818485 2560 generic.go:356] "Generic (PLEG): container finished" podID="d608c8e3-773f-4690-b1c8-ebed82fc79a5" containerID="505edcb44a132b0a17d636e94d3cd017ce8180b07962ebb5c4371cff9bfbe690" exitCode=0 Jul 08 19:11:05.818729 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:05.818587 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s" event={"ID":"d608c8e3-773f-4690-b1c8-ebed82fc79a5","Type":"ContainerDied","Data":"505edcb44a132b0a17d636e94d3cd017ce8180b07962ebb5c4371cff9bfbe690"} Jul 08 19:11:05.818729 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:05.818635 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s" event={"ID":"d608c8e3-773f-4690-b1c8-ebed82fc79a5","Type":"ContainerStarted","Data":"2cf8560c9cdb60de0b5bccd80c25c63e110035843d04880ac8218507bd176cd3"} Jul 08 19:11:06.497346 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:06.497322 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8kkhb"] Jul 08 19:11:06.499486 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:11:06.499460 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb07d0123_7b08_48c6_9de8_5a6c8be269a0.slice/crio-19034715e5ed866c683cb9bef196f8d28b903e4dc3ae2fe124eccc40bec98853 WatchSource:0}: Error finding container 19034715e5ed866c683cb9bef196f8d28b903e4dc3ae2fe124eccc40bec98853: Status 404 returned error can't find the container with id 19034715e5ed866c683cb9bef196f8d28b903e4dc3ae2fe124eccc40bec98853 Jul 08 19:11:06.827605 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:06.827516 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-4mshg" event={"ID":"ecbabcc6-70d7-4ba3-bde5-b0fad9af54bc","Type":"ContainerStarted","Data":"bec9c84b5a56784678267ea2e2f76a212fda8299003bd49922be54b64e4455c6"} Jul 08 19:11:06.827826 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:06.827744 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-4mshg" Jul 08 19:11:06.829572 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:06.829521 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8kkhb" event={"ID":"b07d0123-7b08-48c6-9de8-5a6c8be269a0","Type":"ContainerStarted","Data":"fb53f400d3265d12f2e5b9182251458e5aac0885ba5a95e2bb77152be8f30ce2"} Jul 08 19:11:06.829713 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:06.829584 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8kkhb" event={"ID":"b07d0123-7b08-48c6-9de8-5a6c8be269a0","Type":"ContainerStarted","Data":"19034715e5ed866c683cb9bef196f8d28b903e4dc3ae2fe124eccc40bec98853"} Jul 08 19:11:06.849021 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:06.848970 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-4mshg" podStartSLOduration=1.117091786 podStartE2EDuration="3.848951516s" totalImagesPullingTime="2.73185973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:11:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:06.845730976 +0000 UTC m=+341.023864976" watchObservedRunningTime="2026-07-08 19:11:06.848951516 +0000 UTC m=+341.027085524" Jul 08 19:11:06.864640 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:06.864586 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8kkhb" podStartSLOduration=1.864566897 podStartE2EDuration="1.864566897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:11:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:06.86217247 +0000 UTC m=+341.040306477" watchObservedRunningTime="2026-07-08 19:11:06.864566897 +0000 UTC m=+341.042700905" Jul 08 19:11:08.840398 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:08.840364 2560 generic.go:356] "Generic (PLEG): container finished" podID="d608c8e3-773f-4690-b1c8-ebed82fc79a5" containerID="70c6b22b92897c35338fe4be22b7b1898922a9e63d1537573c1ed08a8756187d" exitCode=0 Jul 08 19:11:08.840882 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:08.840426 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s" event={"ID":"d608c8e3-773f-4690-b1c8-ebed82fc79a5","Type":"ContainerDied","Data":"70c6b22b92897c35338fe4be22b7b1898922a9e63d1537573c1ed08a8756187d"} Jul 08 19:11:09.846627 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:09.846593 2560 generic.go:356] "Generic (PLEG): container finished" podID="d608c8e3-773f-4690-b1c8-ebed82fc79a5" containerID="d6803d5b6a7b5f687c63ace7f20675cfa514297ac8e20741f92f164e5dd06914" exitCode=0 Jul 08 19:11:09.846973 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:09.846660 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s" event={"ID":"d608c8e3-773f-4690-b1c8-ebed82fc79a5","Type":"ContainerDied","Data":"d6803d5b6a7b5f687c63ace7f20675cfa514297ac8e20741f92f164e5dd06914"} Jul 08 19:11:10.973083 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:10.973061 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s" Jul 08 19:11:11.039727 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:11.039696 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d608c8e3-773f-4690-b1c8-ebed82fc79a5-bundle\") pod \"d608c8e3-773f-4690-b1c8-ebed82fc79a5\" (UID: \"d608c8e3-773f-4690-b1c8-ebed82fc79a5\") " Jul 08 19:11:11.039727 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:11.039732 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c5t65\" (UniqueName: \"kubernetes.io/projected/d608c8e3-773f-4690-b1c8-ebed82fc79a5-kube-api-access-c5t65\") pod \"d608c8e3-773f-4690-b1c8-ebed82fc79a5\" (UID: \"d608c8e3-773f-4690-b1c8-ebed82fc79a5\") " Jul 08 19:11:11.039919 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:11.039792 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d608c8e3-773f-4690-b1c8-ebed82fc79a5-util\") pod \"d608c8e3-773f-4690-b1c8-ebed82fc79a5\" (UID: \"d608c8e3-773f-4690-b1c8-ebed82fc79a5\") " Jul 08 19:11:11.040192 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:11.040160 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d608c8e3-773f-4690-b1c8-ebed82fc79a5-bundle" (OuterVolumeSpecName: "bundle") pod "d608c8e3-773f-4690-b1c8-ebed82fc79a5" (UID: "d608c8e3-773f-4690-b1c8-ebed82fc79a5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:11:11.042029 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:11.042005 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d608c8e3-773f-4690-b1c8-ebed82fc79a5-kube-api-access-c5t65" (OuterVolumeSpecName: "kube-api-access-c5t65") pod "d608c8e3-773f-4690-b1c8-ebed82fc79a5" (UID: "d608c8e3-773f-4690-b1c8-ebed82fc79a5"). InnerVolumeSpecName "kube-api-access-c5t65". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:11:11.044190 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:11.044172 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d608c8e3-773f-4690-b1c8-ebed82fc79a5-util" (OuterVolumeSpecName: "util") pod "d608c8e3-773f-4690-b1c8-ebed82fc79a5" (UID: "d608c8e3-773f-4690-b1c8-ebed82fc79a5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:11:11.141335 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:11.141308 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d608c8e3-773f-4690-b1c8-ebed82fc79a5-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:11:11.141464 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:11.141336 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c5t65\" (UniqueName: \"kubernetes.io/projected/d608c8e3-773f-4690-b1c8-ebed82fc79a5-kube-api-access-c5t65\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:11:11.141464 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:11.141351 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d608c8e3-773f-4690-b1c8-ebed82fc79a5-util\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:11:11.857175 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:11.857142 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s" Jul 08 19:11:11.857175 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:11.857147 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fvkl8s" event={"ID":"d608c8e3-773f-4690-b1c8-ebed82fc79a5","Type":"ContainerDied","Data":"2cf8560c9cdb60de0b5bccd80c25c63e110035843d04880ac8218507bd176cd3"} Jul 08 19:11:11.857358 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:11.857209 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2cf8560c9cdb60de0b5bccd80c25c63e110035843d04880ac8218507bd176cd3" Jul 08 19:11:12.835861 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:12.835833 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-4mshg" Jul 08 19:11:18.007390 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.007315 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-426bn"] Jul 08 19:11:18.007827 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.007715 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d608c8e3-773f-4690-b1c8-ebed82fc79a5" containerName="extract" Jul 08 19:11:18.007827 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.007731 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d608c8e3-773f-4690-b1c8-ebed82fc79a5" containerName="extract" Jul 08 19:11:18.007827 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.007747 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d608c8e3-773f-4690-b1c8-ebed82fc79a5" containerName="util" Jul 08 19:11:18.007827 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.007752 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d608c8e3-773f-4690-b1c8-ebed82fc79a5" containerName="util" Jul 08 19:11:18.007827 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.007764 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d608c8e3-773f-4690-b1c8-ebed82fc79a5" containerName="pull" Jul 08 19:11:18.007827 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.007769 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d608c8e3-773f-4690-b1c8-ebed82fc79a5" containerName="pull" Jul 08 19:11:18.007827 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.007818 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d608c8e3-773f-4690-b1c8-ebed82fc79a5" containerName="extract" Jul 08 19:11:18.029249 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.029227 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-426bn"] Jul 08 19:11:18.029381 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.029310 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-426bn" Jul 08 19:11:18.032060 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.032037 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 19:11:18.032174 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.032146 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 19:11:18.032970 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.032952 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-nbvjv\"" Jul 08 19:11:18.098040 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.098012 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzjrq\" (UniqueName: \"kubernetes.io/projected/f62a0f2d-a379-4601-892d-6c9fb10cd3fe-kube-api-access-tzjrq\") pod \"openshift-lws-operator-fd8ccff4c-426bn\" (UID: \"f62a0f2d-a379-4601-892d-6c9fb10cd3fe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-426bn" Jul 08 19:11:18.098040 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.098041 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f62a0f2d-a379-4601-892d-6c9fb10cd3fe-tmp\") pod \"openshift-lws-operator-fd8ccff4c-426bn\" (UID: \"f62a0f2d-a379-4601-892d-6c9fb10cd3fe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-426bn" Jul 08 19:11:18.198695 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.198670 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzjrq\" (UniqueName: \"kubernetes.io/projected/f62a0f2d-a379-4601-892d-6c9fb10cd3fe-kube-api-access-tzjrq\") pod \"openshift-lws-operator-fd8ccff4c-426bn\" (UID: \"f62a0f2d-a379-4601-892d-6c9fb10cd3fe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-426bn" Jul 08 19:11:18.198798 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.198700 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f62a0f2d-a379-4601-892d-6c9fb10cd3fe-tmp\") pod \"openshift-lws-operator-fd8ccff4c-426bn\" (UID: \"f62a0f2d-a379-4601-892d-6c9fb10cd3fe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-426bn" Jul 08 19:11:18.199010 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.198995 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f62a0f2d-a379-4601-892d-6c9fb10cd3fe-tmp\") pod \"openshift-lws-operator-fd8ccff4c-426bn\" (UID: \"f62a0f2d-a379-4601-892d-6c9fb10cd3fe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-426bn" Jul 08 19:11:18.212149 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.212128 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzjrq\" (UniqueName: \"kubernetes.io/projected/f62a0f2d-a379-4601-892d-6c9fb10cd3fe-kube-api-access-tzjrq\") pod \"openshift-lws-operator-fd8ccff4c-426bn\" (UID: \"f62a0f2d-a379-4601-892d-6c9fb10cd3fe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-426bn" Jul 08 19:11:18.338638 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.338557 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-426bn" Jul 08 19:11:18.502874 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.502849 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-426bn"] Jul 08 19:11:18.505252 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:11:18.505215 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf62a0f2d_a379_4601_892d_6c9fb10cd3fe.slice/crio-3931db56b0b69792bc0e28dd9641c05232dd9834d3e8d7d6b15d4f31ff10fff6 WatchSource:0}: Error finding container 3931db56b0b69792bc0e28dd9641c05232dd9834d3e8d7d6b15d4f31ff10fff6: Status 404 returned error can't find the container with id 3931db56b0b69792bc0e28dd9641c05232dd9834d3e8d7d6b15d4f31ff10fff6 Jul 08 19:11:18.893675 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:18.893639 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-426bn" event={"ID":"f62a0f2d-a379-4601-892d-6c9fb10cd3fe","Type":"ContainerStarted","Data":"3931db56b0b69792bc0e28dd9641c05232dd9834d3e8d7d6b15d4f31ff10fff6"} Jul 08 19:11:21.910119 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:21.910084 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-426bn" event={"ID":"f62a0f2d-a379-4601-892d-6c9fb10cd3fe","Type":"ContainerStarted","Data":"31d266c13e1e2294688f4506b260c6f81f90fcb35120d61a02b7045080af2e7c"} Jul 08 19:11:21.927268 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:21.927218 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-426bn" podStartSLOduration=2.359301101 podStartE2EDuration="4.927199481s" totalImagesPullingTime="2.56789838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:11:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:21.927156024 +0000 UTC m=+356.105290029" watchObservedRunningTime="2026-07-08 19:11:21.927199481 +0000 UTC m=+356.105333489" Jul 08 19:11:25.065041 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:25.065010 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k"] Jul 08 19:11:25.067585 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:25.067569 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k" Jul 08 19:11:25.070325 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:25.070284 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 08 19:11:25.070435 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:25.070328 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 08 19:11:25.070594 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:25.070578 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-lmpd8\"" Jul 08 19:11:25.076777 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:25.076756 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k"] Jul 08 19:11:25.158636 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:25.158600 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/159205aa-fdc4-4d1e-9e2b-08440d7883ac-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k\" (UID: \"159205aa-fdc4-4d1e-9e2b-08440d7883ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k" Jul 08 19:11:25.158636 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:25.158641 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm6mj\" (UniqueName: \"kubernetes.io/projected/159205aa-fdc4-4d1e-9e2b-08440d7883ac-kube-api-access-tm6mj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k\" (UID: \"159205aa-fdc4-4d1e-9e2b-08440d7883ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k" Jul 08 19:11:25.158874 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:25.158667 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/159205aa-fdc4-4d1e-9e2b-08440d7883ac-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k\" (UID: \"159205aa-fdc4-4d1e-9e2b-08440d7883ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k" Jul 08 19:11:25.259493 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:25.259457 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/159205aa-fdc4-4d1e-9e2b-08440d7883ac-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k\" (UID: \"159205aa-fdc4-4d1e-9e2b-08440d7883ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k" Jul 08 19:11:25.259579 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:25.259505 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm6mj\" (UniqueName: \"kubernetes.io/projected/159205aa-fdc4-4d1e-9e2b-08440d7883ac-kube-api-access-tm6mj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k\" (UID: \"159205aa-fdc4-4d1e-9e2b-08440d7883ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k" Jul 08 19:11:25.259579 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:25.259566 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/159205aa-fdc4-4d1e-9e2b-08440d7883ac-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k\" (UID: \"159205aa-fdc4-4d1e-9e2b-08440d7883ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k" Jul 08 19:11:25.259970 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:25.259946 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/159205aa-fdc4-4d1e-9e2b-08440d7883ac-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k\" (UID: \"159205aa-fdc4-4d1e-9e2b-08440d7883ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k" Jul 08 19:11:25.260006 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:25.259955 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/159205aa-fdc4-4d1e-9e2b-08440d7883ac-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k\" (UID: \"159205aa-fdc4-4d1e-9e2b-08440d7883ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k" Jul 08 19:11:25.268456 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:25.268431 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm6mj\" (UniqueName: \"kubernetes.io/projected/159205aa-fdc4-4d1e-9e2b-08440d7883ac-kube-api-access-tm6mj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k\" (UID: \"159205aa-fdc4-4d1e-9e2b-08440d7883ac\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k" Jul 08 19:11:25.376695 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:25.376608 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k" Jul 08 19:11:25.506576 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:25.506545 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k"] Jul 08 19:11:25.508951 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:11:25.508929 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod159205aa_fdc4_4d1e_9e2b_08440d7883ac.slice/crio-403e496e8aebcdc6233b704ebe7530ac5ee1904c691832b5b2a0de016c358573 WatchSource:0}: Error finding container 403e496e8aebcdc6233b704ebe7530ac5ee1904c691832b5b2a0de016c358573: Status 404 returned error can't find the container with id 403e496e8aebcdc6233b704ebe7530ac5ee1904c691832b5b2a0de016c358573 Jul 08 19:11:25.928277 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:25.928248 2560 generic.go:356] "Generic (PLEG): container finished" podID="159205aa-fdc4-4d1e-9e2b-08440d7883ac" containerID="59a2cb4feda1568011b0ee50531e79ab75b067a6042e69c691a3f87407e3ff16" exitCode=0 Jul 08 19:11:25.928454 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:25.928342 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k" event={"ID":"159205aa-fdc4-4d1e-9e2b-08440d7883ac","Type":"ContainerDied","Data":"59a2cb4feda1568011b0ee50531e79ab75b067a6042e69c691a3f87407e3ff16"} Jul 08 19:11:25.928454 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:25.928400 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k" event={"ID":"159205aa-fdc4-4d1e-9e2b-08440d7883ac","Type":"ContainerStarted","Data":"403e496e8aebcdc6233b704ebe7530ac5ee1904c691832b5b2a0de016c358573"} Jul 08 19:11:26.934686 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:26.934643 2560 generic.go:356] "Generic (PLEG): container finished" podID="159205aa-fdc4-4d1e-9e2b-08440d7883ac" containerID="f716c94f9aa9633b9ca957b1dc93515631a57ee58dda433430b6e31ef0951c92" exitCode=0 Jul 08 19:11:26.935073 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:26.934738 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k" event={"ID":"159205aa-fdc4-4d1e-9e2b-08440d7883ac","Type":"ContainerDied","Data":"f716c94f9aa9633b9ca957b1dc93515631a57ee58dda433430b6e31ef0951c92"} Jul 08 19:11:27.941617 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:27.941577 2560 generic.go:356] "Generic (PLEG): container finished" podID="159205aa-fdc4-4d1e-9e2b-08440d7883ac" containerID="111cea694412cf84e925b098cd1e922c77caeddb8394093170cd2bb8b1ae8134" exitCode=0 Jul 08 19:11:27.942073 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:27.941669 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k" event={"ID":"159205aa-fdc4-4d1e-9e2b-08440d7883ac","Type":"ContainerDied","Data":"111cea694412cf84e925b098cd1e922c77caeddb8394093170cd2bb8b1ae8134"} Jul 08 19:11:29.070753 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:29.070731 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k" Jul 08 19:11:29.192157 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:29.192127 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/159205aa-fdc4-4d1e-9e2b-08440d7883ac-util\") pod \"159205aa-fdc4-4d1e-9e2b-08440d7883ac\" (UID: \"159205aa-fdc4-4d1e-9e2b-08440d7883ac\") " Jul 08 19:11:29.192157 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:29.192164 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tm6mj\" (UniqueName: \"kubernetes.io/projected/159205aa-fdc4-4d1e-9e2b-08440d7883ac-kube-api-access-tm6mj\") pod \"159205aa-fdc4-4d1e-9e2b-08440d7883ac\" (UID: \"159205aa-fdc4-4d1e-9e2b-08440d7883ac\") " Jul 08 19:11:29.192400 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:29.192192 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/159205aa-fdc4-4d1e-9e2b-08440d7883ac-bundle\") pod \"159205aa-fdc4-4d1e-9e2b-08440d7883ac\" (UID: \"159205aa-fdc4-4d1e-9e2b-08440d7883ac\") " Jul 08 19:11:29.192949 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:29.192923 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/159205aa-fdc4-4d1e-9e2b-08440d7883ac-bundle" (OuterVolumeSpecName: "bundle") pod "159205aa-fdc4-4d1e-9e2b-08440d7883ac" (UID: "159205aa-fdc4-4d1e-9e2b-08440d7883ac"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:11:29.194423 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:29.194397 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/159205aa-fdc4-4d1e-9e2b-08440d7883ac-kube-api-access-tm6mj" (OuterVolumeSpecName: "kube-api-access-tm6mj") pod "159205aa-fdc4-4d1e-9e2b-08440d7883ac" (UID: "159205aa-fdc4-4d1e-9e2b-08440d7883ac"). InnerVolumeSpecName "kube-api-access-tm6mj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:11:29.197567 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:29.197504 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/159205aa-fdc4-4d1e-9e2b-08440d7883ac-util" (OuterVolumeSpecName: "util") pod "159205aa-fdc4-4d1e-9e2b-08440d7883ac" (UID: "159205aa-fdc4-4d1e-9e2b-08440d7883ac"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:11:29.293399 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:29.293355 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/159205aa-fdc4-4d1e-9e2b-08440d7883ac-util\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:11:29.293399 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:29.293390 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tm6mj\" (UniqueName: \"kubernetes.io/projected/159205aa-fdc4-4d1e-9e2b-08440d7883ac-kube-api-access-tm6mj\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:11:29.293399 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:29.293400 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/159205aa-fdc4-4d1e-9e2b-08440d7883ac-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:11:29.953184 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:29.953156 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k" Jul 08 19:11:29.953359 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:29.953150 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afmrc6k" event={"ID":"159205aa-fdc4-4d1e-9e2b-08440d7883ac","Type":"ContainerDied","Data":"403e496e8aebcdc6233b704ebe7530ac5ee1904c691832b5b2a0de016c358573"} Jul 08 19:11:29.953359 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:29.953295 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="403e496e8aebcdc6233b704ebe7530ac5ee1904c691832b5b2a0de016c358573" Jul 08 19:11:37.602731 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.602698 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw"] Jul 08 19:11:37.603093 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.603019 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="159205aa-fdc4-4d1e-9e2b-08440d7883ac" containerName="extract" Jul 08 19:11:37.603093 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.603031 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="159205aa-fdc4-4d1e-9e2b-08440d7883ac" containerName="extract" Jul 08 19:11:37.603093 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.603039 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="159205aa-fdc4-4d1e-9e2b-08440d7883ac" containerName="pull" Jul 08 19:11:37.603093 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.603044 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="159205aa-fdc4-4d1e-9e2b-08440d7883ac" containerName="pull" Jul 08 19:11:37.603093 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.603051 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="159205aa-fdc4-4d1e-9e2b-08440d7883ac" containerName="util" Jul 08 19:11:37.603093 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.603057 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="159205aa-fdc4-4d1e-9e2b-08440d7883ac" containerName="util" Jul 08 19:11:37.603285 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.603116 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="159205aa-fdc4-4d1e-9e2b-08440d7883ac" containerName="extract" Jul 08 19:11:37.605657 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.605642 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" Jul 08 19:11:37.608313 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.608289 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 19:11:37.608439 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.608336 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 19:11:37.608511 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.608473 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 19:11:37.608640 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.608620 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-fl96f\"" Jul 08 19:11:37.609057 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.609041 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 19:11:37.618898 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.618879 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw"] Jul 08 19:11:37.657476 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.657456 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/442cb34b-f9b9-4ed7-87a4-0199a56cb2a9-manifests\") pod \"opendatahub-operator-controller-manager-6667fbd97f-b2kdw\" (UID: \"442cb34b-f9b9-4ed7-87a4-0199a56cb2a9\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" Jul 08 19:11:37.657589 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.657485 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/442cb34b-f9b9-4ed7-87a4-0199a56cb2a9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6667fbd97f-b2kdw\" (UID: \"442cb34b-f9b9-4ed7-87a4-0199a56cb2a9\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" Jul 08 19:11:37.657589 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.657562 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhj5k\" (UniqueName: \"kubernetes.io/projected/442cb34b-f9b9-4ed7-87a4-0199a56cb2a9-kube-api-access-qhj5k\") pod \"opendatahub-operator-controller-manager-6667fbd97f-b2kdw\" (UID: \"442cb34b-f9b9-4ed7-87a4-0199a56cb2a9\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" Jul 08 19:11:37.657589 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.657581 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/442cb34b-f9b9-4ed7-87a4-0199a56cb2a9-webhook-cert\") pod \"opendatahub-operator-controller-manager-6667fbd97f-b2kdw\" (UID: \"442cb34b-f9b9-4ed7-87a4-0199a56cb2a9\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" Jul 08 19:11:37.657700 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.657608 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/442cb34b-f9b9-4ed7-87a4-0199a56cb2a9-tmp\") pod \"opendatahub-operator-controller-manager-6667fbd97f-b2kdw\" (UID: \"442cb34b-f9b9-4ed7-87a4-0199a56cb2a9\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" Jul 08 19:11:37.758135 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.758112 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/442cb34b-f9b9-4ed7-87a4-0199a56cb2a9-tmp\") pod \"opendatahub-operator-controller-manager-6667fbd97f-b2kdw\" (UID: \"442cb34b-f9b9-4ed7-87a4-0199a56cb2a9\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" Jul 08 19:11:37.758251 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.758147 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/442cb34b-f9b9-4ed7-87a4-0199a56cb2a9-manifests\") pod \"opendatahub-operator-controller-manager-6667fbd97f-b2kdw\" (UID: \"442cb34b-f9b9-4ed7-87a4-0199a56cb2a9\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" Jul 08 19:11:37.758251 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.758168 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/442cb34b-f9b9-4ed7-87a4-0199a56cb2a9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6667fbd97f-b2kdw\" (UID: \"442cb34b-f9b9-4ed7-87a4-0199a56cb2a9\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" Jul 08 19:11:37.758251 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.758217 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhj5k\" (UniqueName: \"kubernetes.io/projected/442cb34b-f9b9-4ed7-87a4-0199a56cb2a9-kube-api-access-qhj5k\") pod \"opendatahub-operator-controller-manager-6667fbd97f-b2kdw\" (UID: \"442cb34b-f9b9-4ed7-87a4-0199a56cb2a9\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" Jul 08 19:11:37.758360 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.758312 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/442cb34b-f9b9-4ed7-87a4-0199a56cb2a9-webhook-cert\") pod \"opendatahub-operator-controller-manager-6667fbd97f-b2kdw\" (UID: \"442cb34b-f9b9-4ed7-87a4-0199a56cb2a9\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" Jul 08 19:11:37.758601 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.758575 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/442cb34b-f9b9-4ed7-87a4-0199a56cb2a9-manifests\") pod \"opendatahub-operator-controller-manager-6667fbd97f-b2kdw\" (UID: \"442cb34b-f9b9-4ed7-87a4-0199a56cb2a9\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" Jul 08 19:11:37.760653 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.760631 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/442cb34b-f9b9-4ed7-87a4-0199a56cb2a9-tmp\") pod \"opendatahub-operator-controller-manager-6667fbd97f-b2kdw\" (UID: \"442cb34b-f9b9-4ed7-87a4-0199a56cb2a9\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" Jul 08 19:11:37.760760 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.760714 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/442cb34b-f9b9-4ed7-87a4-0199a56cb2a9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6667fbd97f-b2kdw\" (UID: \"442cb34b-f9b9-4ed7-87a4-0199a56cb2a9\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" Jul 08 19:11:37.760819 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.760760 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/442cb34b-f9b9-4ed7-87a4-0199a56cb2a9-webhook-cert\") pod \"opendatahub-operator-controller-manager-6667fbd97f-b2kdw\" (UID: \"442cb34b-f9b9-4ed7-87a4-0199a56cb2a9\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" Jul 08 19:11:37.773081 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.773051 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhj5k\" (UniqueName: \"kubernetes.io/projected/442cb34b-f9b9-4ed7-87a4-0199a56cb2a9-kube-api-access-qhj5k\") pod \"opendatahub-operator-controller-manager-6667fbd97f-b2kdw\" (UID: \"442cb34b-f9b9-4ed7-87a4-0199a56cb2a9\") " pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" Jul 08 19:11:37.915763 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:37.915732 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" Jul 08 19:11:38.060919 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:38.060893 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw"] Jul 08 19:11:38.064300 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:11:38.064277 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod442cb34b_f9b9_4ed7_87a4_0199a56cb2a9.slice/crio-bd99f1b55837bfd5dd33ff22beceddfbdd2dbd5b5867351ec96b0b9979c9ffd4 WatchSource:0}: Error finding container bd99f1b55837bfd5dd33ff22beceddfbdd2dbd5b5867351ec96b0b9979c9ffd4: Status 404 returned error can't find the container with id bd99f1b55837bfd5dd33ff22beceddfbdd2dbd5b5867351ec96b0b9979c9ffd4 Jul 08 19:11:38.997900 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:38.997821 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" event={"ID":"442cb34b-f9b9-4ed7-87a4-0199a56cb2a9","Type":"ContainerStarted","Data":"bd99f1b55837bfd5dd33ff22beceddfbdd2dbd5b5867351ec96b0b9979c9ffd4"} Jul 08 19:11:42.015196 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:42.015147 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" event={"ID":"442cb34b-f9b9-4ed7-87a4-0199a56cb2a9","Type":"ContainerStarted","Data":"38b2e7e6552bd14f72df575ddc2e0408ad0552a0aac2e44535a4034a58fd21c6"} Jul 08 19:11:43.021032 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:43.021001 2560 generic.go:356] "Generic (PLEG): container finished" podID="442cb34b-f9b9-4ed7-87a4-0199a56cb2a9" containerID="38b2e7e6552bd14f72df575ddc2e0408ad0552a0aac2e44535a4034a58fd21c6" exitCode=0 Jul 08 19:11:43.021419 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:43.021085 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" event={"ID":"442cb34b-f9b9-4ed7-87a4-0199a56cb2a9","Type":"ContainerDied","Data":"38b2e7e6552bd14f72df575ddc2e0408ad0552a0aac2e44535a4034a58fd21c6"} Jul 08 19:11:44.027598 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:44.027565 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" event={"ID":"442cb34b-f9b9-4ed7-87a4-0199a56cb2a9","Type":"ContainerStarted","Data":"fad7d981a1420174cb781f863b82ed770fca1d9ee6d1d2714c9f1acc32b1f6fe"} Jul 08 19:11:44.027974 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:44.027677 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" Jul 08 19:11:55.035130 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:55.035099 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" Jul 08 19:11:55.083409 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:55.083359 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6667fbd97f-b2kdw" podStartSLOduration=13.540329457 podStartE2EDuration="18.083343767s" totalImagesPullingTime="3.512185067s" totalInitContainerRuntime="1.030829243s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:11:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:11:44.084923049 +0000 UTC m=+378.263057057" watchObservedRunningTime="2026-07-08 19:11:55.083343767 +0000 UTC m=+389.261477774" Jul 08 19:11:59.246240 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.246197 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-565878cbdc-lqb2c"] Jul 08 19:11:59.253128 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.253110 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-565878cbdc-lqb2c" Jul 08 19:11:59.264821 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.264799 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 19:11:59.264927 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.264799 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-cr7zp\"" Jul 08 19:11:59.265195 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.265182 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 19:11:59.265437 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.265425 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 08 19:11:59.269350 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.269337 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 19:11:59.288233 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.288207 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-565878cbdc-lqb2c"] Jul 08 19:11:59.361848 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.361819 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pnwjc"] Jul 08 19:11:59.364371 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.364357 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pnwjc" Jul 08 19:11:59.367610 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.367591 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 08 19:11:59.367835 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.367819 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-7t5rq\"" Jul 08 19:11:59.395037 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.395010 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pnwjc"] Jul 08 19:11:59.430644 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.430617 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06db37a8-78bb-47af-8d73-815c55f9f88a-tls-certs\") pod \"kube-auth-proxy-565878cbdc-lqb2c\" (UID: \"06db37a8-78bb-47af-8d73-815c55f9f88a\") " pod="openshift-ingress/kube-auth-proxy-565878cbdc-lqb2c" Jul 08 19:11:59.430644 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.430645 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06db37a8-78bb-47af-8d73-815c55f9f88a-tmp\") pod \"kube-auth-proxy-565878cbdc-lqb2c\" (UID: \"06db37a8-78bb-47af-8d73-815c55f9f88a\") " pod="openshift-ingress/kube-auth-proxy-565878cbdc-lqb2c" Jul 08 19:11:59.430878 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.430673 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htrcm\" (UniqueName: \"kubernetes.io/projected/06db37a8-78bb-47af-8d73-815c55f9f88a-kube-api-access-htrcm\") pod \"kube-auth-proxy-565878cbdc-lqb2c\" (UID: \"06db37a8-78bb-47af-8d73-815c55f9f88a\") " pod="openshift-ingress/kube-auth-proxy-565878cbdc-lqb2c" Jul 08 19:11:59.531938 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.531852 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06db37a8-78bb-47af-8d73-815c55f9f88a-tls-certs\") pod \"kube-auth-proxy-565878cbdc-lqb2c\" (UID: \"06db37a8-78bb-47af-8d73-815c55f9f88a\") " pod="openshift-ingress/kube-auth-proxy-565878cbdc-lqb2c" Jul 08 19:11:59.531938 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.531886 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06db37a8-78bb-47af-8d73-815c55f9f88a-tmp\") pod \"kube-auth-proxy-565878cbdc-lqb2c\" (UID: \"06db37a8-78bb-47af-8d73-815c55f9f88a\") " pod="openshift-ingress/kube-auth-proxy-565878cbdc-lqb2c" Jul 08 19:11:59.531938 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.531912 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65100a2c-1816-4008-a01b-b4761767a40d-cert\") pod \"odh-model-controller-7998c67dc4-pnwjc\" (UID: \"65100a2c-1816-4008-a01b-b4761767a40d\") " pod="opendatahub/odh-model-controller-7998c67dc4-pnwjc" Jul 08 19:11:59.532206 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.531950 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htrcm\" (UniqueName: \"kubernetes.io/projected/06db37a8-78bb-47af-8d73-815c55f9f88a-kube-api-access-htrcm\") pod \"kube-auth-proxy-565878cbdc-lqb2c\" (UID: \"06db37a8-78bb-47af-8d73-815c55f9f88a\") " pod="openshift-ingress/kube-auth-proxy-565878cbdc-lqb2c" Jul 08 19:11:59.532206 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.532101 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25fv6\" (UniqueName: \"kubernetes.io/projected/65100a2c-1816-4008-a01b-b4761767a40d-kube-api-access-25fv6\") pod \"odh-model-controller-7998c67dc4-pnwjc\" (UID: \"65100a2c-1816-4008-a01b-b4761767a40d\") " pod="opendatahub/odh-model-controller-7998c67dc4-pnwjc" Jul 08 19:11:59.534201 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.534183 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06db37a8-78bb-47af-8d73-815c55f9f88a-tmp\") pod \"kube-auth-proxy-565878cbdc-lqb2c\" (UID: \"06db37a8-78bb-47af-8d73-815c55f9f88a\") " pod="openshift-ingress/kube-auth-proxy-565878cbdc-lqb2c" Jul 08 19:11:59.534401 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.534384 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06db37a8-78bb-47af-8d73-815c55f9f88a-tls-certs\") pod \"kube-auth-proxy-565878cbdc-lqb2c\" (UID: \"06db37a8-78bb-47af-8d73-815c55f9f88a\") " pod="openshift-ingress/kube-auth-proxy-565878cbdc-lqb2c" Jul 08 19:11:59.576608 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.576573 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htrcm\" (UniqueName: \"kubernetes.io/projected/06db37a8-78bb-47af-8d73-815c55f9f88a-kube-api-access-htrcm\") pod \"kube-auth-proxy-565878cbdc-lqb2c\" (UID: \"06db37a8-78bb-47af-8d73-815c55f9f88a\") " pod="openshift-ingress/kube-auth-proxy-565878cbdc-lqb2c" Jul 08 19:11:59.632490 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.632464 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25fv6\" (UniqueName: \"kubernetes.io/projected/65100a2c-1816-4008-a01b-b4761767a40d-kube-api-access-25fv6\") pod \"odh-model-controller-7998c67dc4-pnwjc\" (UID: \"65100a2c-1816-4008-a01b-b4761767a40d\") " pod="opendatahub/odh-model-controller-7998c67dc4-pnwjc" Jul 08 19:11:59.632714 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.632508 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65100a2c-1816-4008-a01b-b4761767a40d-cert\") pod \"odh-model-controller-7998c67dc4-pnwjc\" (UID: \"65100a2c-1816-4008-a01b-b4761767a40d\") " pod="opendatahub/odh-model-controller-7998c67dc4-pnwjc" Jul 08 19:11:59.632714 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:11:59.632654 2560 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 08 19:11:59.632714 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:11:59.632714 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65100a2c-1816-4008-a01b-b4761767a40d-cert podName:65100a2c-1816-4008-a01b-b4761767a40d nodeName:}" failed. No retries permitted until 2026-07-08 19:12:00.132697387 +0000 UTC m=+394.310831371 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/65100a2c-1816-4008-a01b-b4761767a40d-cert") pod "odh-model-controller-7998c67dc4-pnwjc" (UID: "65100a2c-1816-4008-a01b-b4761767a40d") : secret "odh-model-controller-webhook-cert" not found Jul 08 19:11:59.666779 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.666751 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25fv6\" (UniqueName: \"kubernetes.io/projected/65100a2c-1816-4008-a01b-b4761767a40d-kube-api-access-25fv6\") pod \"odh-model-controller-7998c67dc4-pnwjc\" (UID: \"65100a2c-1816-4008-a01b-b4761767a40d\") " pod="opendatahub/odh-model-controller-7998c67dc4-pnwjc" Jul 08 19:11:59.861930 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:11:59.861848 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-565878cbdc-lqb2c" Jul 08 19:12:00.017510 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:00.017483 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-565878cbdc-lqb2c"] Jul 08 19:12:00.019392 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:12:00.019366 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06db37a8_78bb_47af_8d73_815c55f9f88a.slice/crio-9222184012f96d7a48cd238cb0fe216ecdd5d01b6eb40e0021ae8bfa28043799 WatchSource:0}: Error finding container 9222184012f96d7a48cd238cb0fe216ecdd5d01b6eb40e0021ae8bfa28043799: Status 404 returned error can't find the container with id 9222184012f96d7a48cd238cb0fe216ecdd5d01b6eb40e0021ae8bfa28043799 Jul 08 19:12:00.107797 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:00.107768 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-565878cbdc-lqb2c" event={"ID":"06db37a8-78bb-47af-8d73-815c55f9f88a","Type":"ContainerStarted","Data":"9222184012f96d7a48cd238cb0fe216ecdd5d01b6eb40e0021ae8bfa28043799"} Jul 08 19:12:00.138195 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:00.138178 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65100a2c-1816-4008-a01b-b4761767a40d-cert\") pod \"odh-model-controller-7998c67dc4-pnwjc\" (UID: \"65100a2c-1816-4008-a01b-b4761767a40d\") " pod="opendatahub/odh-model-controller-7998c67dc4-pnwjc" Jul 08 19:12:00.140638 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:00.140618 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/65100a2c-1816-4008-a01b-b4761767a40d-cert\") pod \"odh-model-controller-7998c67dc4-pnwjc\" (UID: \"65100a2c-1816-4008-a01b-b4761767a40d\") " pod="opendatahub/odh-model-controller-7998c67dc4-pnwjc" Jul 08 19:12:00.273926 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:00.273899 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pnwjc" Jul 08 19:12:00.457634 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:00.457606 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pnwjc"] Jul 08 19:12:00.459401 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:12:00.459374 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65100a2c_1816_4008_a01b_b4761767a40d.slice/crio-8b929c69dfc27124332de1132459f131f2dde769be0b8a31e90d308b3b8cd8ca WatchSource:0}: Error finding container 8b929c69dfc27124332de1132459f131f2dde769be0b8a31e90d308b3b8cd8ca: Status 404 returned error can't find the container with id 8b929c69dfc27124332de1132459f131f2dde769be0b8a31e90d308b3b8cd8ca Jul 08 19:12:01.011777 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:01.011740 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-p2b8c"] Jul 08 19:12:01.014630 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:01.014607 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-p2b8c" Jul 08 19:12:01.018222 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:01.018198 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 08 19:12:01.040143 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:01.040090 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-p2b8c"] Jul 08 19:12:01.115915 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:01.115865 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pnwjc" event={"ID":"65100a2c-1816-4008-a01b-b4761767a40d","Type":"ContainerStarted","Data":"8b929c69dfc27124332de1132459f131f2dde769be0b8a31e90d308b3b8cd8ca"} Jul 08 19:12:01.147062 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:01.147025 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9987fb5f-d765-4f19-b236-039ec3cd50aa-tls-certs\") pod \"model-serving-api-54d89bd5fc-p2b8c\" (UID: \"9987fb5f-d765-4f19-b236-039ec3cd50aa\") " pod="opendatahub/model-serving-api-54d89bd5fc-p2b8c" Jul 08 19:12:01.147302 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:01.147132 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58fmw\" (UniqueName: \"kubernetes.io/projected/9987fb5f-d765-4f19-b236-039ec3cd50aa-kube-api-access-58fmw\") pod \"model-serving-api-54d89bd5fc-p2b8c\" (UID: \"9987fb5f-d765-4f19-b236-039ec3cd50aa\") " pod="opendatahub/model-serving-api-54d89bd5fc-p2b8c" Jul 08 19:12:01.248040 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:01.248002 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9987fb5f-d765-4f19-b236-039ec3cd50aa-tls-certs\") pod \"model-serving-api-54d89bd5fc-p2b8c\" (UID: \"9987fb5f-d765-4f19-b236-039ec3cd50aa\") " pod="opendatahub/model-serving-api-54d89bd5fc-p2b8c" Jul 08 19:12:01.248222 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:01.248100 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58fmw\" (UniqueName: \"kubernetes.io/projected/9987fb5f-d765-4f19-b236-039ec3cd50aa-kube-api-access-58fmw\") pod \"model-serving-api-54d89bd5fc-p2b8c\" (UID: \"9987fb5f-d765-4f19-b236-039ec3cd50aa\") " pod="opendatahub/model-serving-api-54d89bd5fc-p2b8c" Jul 08 19:12:01.251200 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:01.251169 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9987fb5f-d765-4f19-b236-039ec3cd50aa-tls-certs\") pod \"model-serving-api-54d89bd5fc-p2b8c\" (UID: \"9987fb5f-d765-4f19-b236-039ec3cd50aa\") " pod="opendatahub/model-serving-api-54d89bd5fc-p2b8c" Jul 08 19:12:01.263261 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:01.263188 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58fmw\" (UniqueName: \"kubernetes.io/projected/9987fb5f-d765-4f19-b236-039ec3cd50aa-kube-api-access-58fmw\") pod \"model-serving-api-54d89bd5fc-p2b8c\" (UID: \"9987fb5f-d765-4f19-b236-039ec3cd50aa\") " pod="opendatahub/model-serving-api-54d89bd5fc-p2b8c" Jul 08 19:12:01.328088 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:01.327828 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-p2b8c" Jul 08 19:12:01.522253 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:01.522211 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-p2b8c"] Jul 08 19:12:01.525721 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:12:01.525691 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9987fb5f_d765_4f19_b236_039ec3cd50aa.slice/crio-96d6814b2c80956444c7ce9231fe56a1c1ebfa41bed18e2ea330e391d4570dfa WatchSource:0}: Error finding container 96d6814b2c80956444c7ce9231fe56a1c1ebfa41bed18e2ea330e391d4570dfa: Status 404 returned error can't find the container with id 96d6814b2c80956444c7ce9231fe56a1c1ebfa41bed18e2ea330e391d4570dfa Jul 08 19:12:02.122893 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:02.122845 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-p2b8c" event={"ID":"9987fb5f-d765-4f19-b236-039ec3cd50aa","Type":"ContainerStarted","Data":"96d6814b2c80956444c7ce9231fe56a1c1ebfa41bed18e2ea330e391d4570dfa"} Jul 08 19:12:05.142907 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:05.142872 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-565878cbdc-lqb2c" event={"ID":"06db37a8-78bb-47af-8d73-815c55f9f88a","Type":"ContainerStarted","Data":"423d7f9b68eac72ef50509eb61f47818d0b44114bbd85fe40d870033ffcce174"} Jul 08 19:12:05.144375 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:05.144349 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-p2b8c" event={"ID":"9987fb5f-d765-4f19-b236-039ec3cd50aa","Type":"ContainerStarted","Data":"9b938a6f1c8dbbbeafc5e2be97feaae9cbd5e65e095971e48f09c998600d0f9b"} Jul 08 19:12:05.144499 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:05.144472 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-p2b8c" Jul 08 19:12:05.145656 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:05.145637 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pnwjc" event={"ID":"65100a2c-1816-4008-a01b-b4761767a40d","Type":"ContainerStarted","Data":"9111b9b4b6774d024e02688f12a6af2b7304a614ff9cf851e6a483e227f768cb"} Jul 08 19:12:05.145777 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:05.145764 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-pnwjc" Jul 08 19:12:05.184822 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:05.184782 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-565878cbdc-lqb2c" podStartSLOduration=1.824187827 podStartE2EDuration="6.184769835s" totalImagesPullingTime="4.360582008s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:11:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:05.18403376 +0000 UTC m=+399.362167766" watchObservedRunningTime="2026-07-08 19:12:05.184769835 +0000 UTC m=+399.362903841" Jul 08 19:12:05.220794 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:05.220747 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-p2b8c" podStartSLOduration=2.374301195 podStartE2EDuration="5.220731345s" totalImagesPullingTime="2.84643015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:05.218854398 +0000 UTC m=+399.396988405" watchObservedRunningTime="2026-07-08 19:12:05.220731345 +0000 UTC m=+399.398865350" Jul 08 19:12:08.274288 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.274237 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-pnwjc" podStartSLOduration=5.363632823 podStartE2EDuration="9.274217934s" totalImagesPullingTime="3.910585111s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:11:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:05.254269542 +0000 UTC m=+399.432403547" watchObservedRunningTime="2026-07-08 19:12:08.274217934 +0000 UTC m=+402.452351969" Jul 08 19:12:08.276047 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.276020 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz"] Jul 08 19:12:08.278878 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.278858 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.288944 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.288924 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 08 19:12:08.289047 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.288966 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-nr2xh\"" Jul 08 19:12:08.300579 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.300558 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz"] Jul 08 19:12:08.308899 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.308874 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b415f298-0354-4f5a-9204-0dfcced0e3c7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.308979 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.308912 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b415f298-0354-4f5a-9204-0dfcced0e3c7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.308979 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.308933 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b415f298-0354-4f5a-9204-0dfcced0e3c7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.309084 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.308978 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmp74\" (UniqueName: \"kubernetes.io/projected/b415f298-0354-4f5a-9204-0dfcced0e3c7-kube-api-access-xmp74\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.309084 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.309005 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b415f298-0354-4f5a-9204-0dfcced0e3c7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.309196 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.309078 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b415f298-0354-4f5a-9204-0dfcced0e3c7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.309196 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.309119 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b415f298-0354-4f5a-9204-0dfcced0e3c7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.309196 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.309160 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b415f298-0354-4f5a-9204-0dfcced0e3c7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.309321 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.309218 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b415f298-0354-4f5a-9204-0dfcced0e3c7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.410328 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.410301 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmp74\" (UniqueName: \"kubernetes.io/projected/b415f298-0354-4f5a-9204-0dfcced0e3c7-kube-api-access-xmp74\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.410328 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.410332 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b415f298-0354-4f5a-9204-0dfcced0e3c7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.410561 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.410355 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b415f298-0354-4f5a-9204-0dfcced0e3c7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.410561 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.410416 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b415f298-0354-4f5a-9204-0dfcced0e3c7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.410561 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.410455 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b415f298-0354-4f5a-9204-0dfcced0e3c7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.410561 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.410500 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b415f298-0354-4f5a-9204-0dfcced0e3c7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.410779 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.410577 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b415f298-0354-4f5a-9204-0dfcced0e3c7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.410779 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.410594 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b415f298-0354-4f5a-9204-0dfcced0e3c7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.410779 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.410612 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b415f298-0354-4f5a-9204-0dfcced0e3c7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.410986 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.410778 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b415f298-0354-4f5a-9204-0dfcced0e3c7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.410986 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.410816 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b415f298-0354-4f5a-9204-0dfcced0e3c7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.410986 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.410894 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b415f298-0354-4f5a-9204-0dfcced0e3c7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.410986 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.410916 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b415f298-0354-4f5a-9204-0dfcced0e3c7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.411345 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.411229 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b415f298-0354-4f5a-9204-0dfcced0e3c7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.413040 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.413020 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b415f298-0354-4f5a-9204-0dfcced0e3c7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.413132 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.413075 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b415f298-0354-4f5a-9204-0dfcced0e3c7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.429824 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.429804 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmp74\" (UniqueName: \"kubernetes.io/projected/b415f298-0354-4f5a-9204-0dfcced0e3c7-kube-api-access-xmp74\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.436362 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.436342 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b415f298-0354-4f5a-9204-0dfcced0e3c7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4bd4zz\" (UID: \"b415f298-0354-4f5a-9204-0dfcced0e3c7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.453002 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.452982 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5"] Jul 08 19:12:08.456348 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.456334 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5" Jul 08 19:12:08.467440 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.467423 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-q52kd\"" Jul 08 19:12:08.467683 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.467663 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 19:12:08.467796 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.467777 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 19:12:08.467875 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.467862 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 19:12:08.496506 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.496479 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5"] Jul 08 19:12:08.511496 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.511475 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/07293172-4f4a-4ebd-9fc1-89503717a147-manager-config\") pod \"lws-controller-manager-6f5b6f7879-lrlh5\" (UID: \"07293172-4f4a-4ebd-9fc1-89503717a147\") " pod="openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5" Jul 08 19:12:08.511633 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.511515 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07293172-4f4a-4ebd-9fc1-89503717a147-cert\") pod \"lws-controller-manager-6f5b6f7879-lrlh5\" (UID: \"07293172-4f4a-4ebd-9fc1-89503717a147\") " pod="openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5" Jul 08 19:12:08.511683 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.511661 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87tct\" (UniqueName: \"kubernetes.io/projected/07293172-4f4a-4ebd-9fc1-89503717a147-kube-api-access-87tct\") pod \"lws-controller-manager-6f5b6f7879-lrlh5\" (UID: \"07293172-4f4a-4ebd-9fc1-89503717a147\") " pod="openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5" Jul 08 19:12:08.511719 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.511694 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/07293172-4f4a-4ebd-9fc1-89503717a147-metrics-cert\") pod \"lws-controller-manager-6f5b6f7879-lrlh5\" (UID: \"07293172-4f4a-4ebd-9fc1-89503717a147\") " pod="openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5" Jul 08 19:12:08.589312 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.589246 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:08.612613 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.612579 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87tct\" (UniqueName: \"kubernetes.io/projected/07293172-4f4a-4ebd-9fc1-89503717a147-kube-api-access-87tct\") pod \"lws-controller-manager-6f5b6f7879-lrlh5\" (UID: \"07293172-4f4a-4ebd-9fc1-89503717a147\") " pod="openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5" Jul 08 19:12:08.612805 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.612788 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/07293172-4f4a-4ebd-9fc1-89503717a147-metrics-cert\") pod \"lws-controller-manager-6f5b6f7879-lrlh5\" (UID: \"07293172-4f4a-4ebd-9fc1-89503717a147\") " pod="openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5" Jul 08 19:12:08.612966 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.612952 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/07293172-4f4a-4ebd-9fc1-89503717a147-manager-config\") pod \"lws-controller-manager-6f5b6f7879-lrlh5\" (UID: \"07293172-4f4a-4ebd-9fc1-89503717a147\") " pod="openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5" Jul 08 19:12:08.613088 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.613074 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07293172-4f4a-4ebd-9fc1-89503717a147-cert\") pod \"lws-controller-manager-6f5b6f7879-lrlh5\" (UID: \"07293172-4f4a-4ebd-9fc1-89503717a147\") " pod="openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5" Jul 08 19:12:08.618439 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.616328 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/07293172-4f4a-4ebd-9fc1-89503717a147-cert\") pod \"lws-controller-manager-6f5b6f7879-lrlh5\" (UID: \"07293172-4f4a-4ebd-9fc1-89503717a147\") " pod="openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5" Jul 08 19:12:08.618439 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.616955 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/07293172-4f4a-4ebd-9fc1-89503717a147-manager-config\") pod \"lws-controller-manager-6f5b6f7879-lrlh5\" (UID: \"07293172-4f4a-4ebd-9fc1-89503717a147\") " pod="openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5" Jul 08 19:12:08.620671 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.620648 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/07293172-4f4a-4ebd-9fc1-89503717a147-metrics-cert\") pod \"lws-controller-manager-6f5b6f7879-lrlh5\" (UID: \"07293172-4f4a-4ebd-9fc1-89503717a147\") " pod="openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5" Jul 08 19:12:08.658861 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.658832 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87tct\" (UniqueName: \"kubernetes.io/projected/07293172-4f4a-4ebd-9fc1-89503717a147-kube-api-access-87tct\") pod \"lws-controller-manager-6f5b6f7879-lrlh5\" (UID: \"07293172-4f4a-4ebd-9fc1-89503717a147\") " pod="openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5" Jul 08 19:12:08.766167 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.766131 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5" Jul 08 19:12:08.800723 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:08.800690 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz"] Jul 08 19:12:08.801782 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:12:08.801754 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb415f298_0354_4f5a_9204_0dfcced0e3c7.slice/crio-5fec80fe38e4e1fdc8fc23992a8420a68b6cae5080ea4b11df23ba25ad1238b7 WatchSource:0}: Error finding container 5fec80fe38e4e1fdc8fc23992a8420a68b6cae5080ea4b11df23ba25ad1238b7: Status 404 returned error can't find the container with id 5fec80fe38e4e1fdc8fc23992a8420a68b6cae5080ea4b11df23ba25ad1238b7 Jul 08 19:12:08.987659 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:12:08.987633 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07293172_4f4a_4ebd_9fc1_89503717a147.slice/crio-6d055b6c777ee4af40fecf242c098dfbf3c805108f74952170d29510e2c792cc WatchSource:0}: Error finding container 6d055b6c777ee4af40fecf242c098dfbf3c805108f74952170d29510e2c792cc: Status 404 returned error can't find the container with id 6d055b6c777ee4af40fecf242c098dfbf3c805108f74952170d29510e2c792cc Jul 08 19:12:09.001640 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:09.001614 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5"] Jul 08 19:12:09.166505 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:09.166473 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" event={"ID":"b415f298-0354-4f5a-9204-0dfcced0e3c7","Type":"ContainerStarted","Data":"5fec80fe38e4e1fdc8fc23992a8420a68b6cae5080ea4b11df23ba25ad1238b7"} Jul 08 19:12:09.167454 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:09.167436 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5" event={"ID":"07293172-4f4a-4ebd-9fc1-89503717a147","Type":"ContainerStarted","Data":"6d055b6c777ee4af40fecf242c098dfbf3c805108f74952170d29510e2c792cc"} Jul 08 19:12:11.707693 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:11.707654 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:12:11.708036 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:11.707725 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:12:11.708036 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:11.707754 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 08 19:12:12.188792 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:12.188744 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" event={"ID":"b415f298-0354-4f5a-9204-0dfcced0e3c7","Type":"ContainerStarted","Data":"3799e0828ae0ac08c466a19a682b4ac6d77e66dd0159c51558242a350ab3ea7e"} Jul 08 19:12:12.190388 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:12.190354 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5" event={"ID":"07293172-4f4a-4ebd-9fc1-89503717a147","Type":"ContainerStarted","Data":"8bf2697e72209c08e5b1580ef922ff9341bd3907c888f29be01f6d9b478e7534"} Jul 08 19:12:12.190514 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:12.190480 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5" Jul 08 19:12:12.224807 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:12.224752 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" podStartSLOduration=1.321163096 podStartE2EDuration="4.224732693s" totalImagesPullingTime="2.903569597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:12.223504052 +0000 UTC m=+406.401638059" watchObservedRunningTime="2026-07-08 19:12:12.224732693 +0000 UTC m=+406.402866700" Jul 08 19:12:12.268681 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:12.268628 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5" podStartSLOduration=1.555714296 podStartE2EDuration="4.268613311s" totalImagesPullingTime="2.712899015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:12:12.267744174 +0000 UTC m=+406.445878183" watchObservedRunningTime="2026-07-08 19:12:12.268613311 +0000 UTC m=+406.446747317" Jul 08 19:12:12.589834 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:12.589747 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:12.594405 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:12.594380 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:13.195017 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:13.194977 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:13.196100 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:13.196079 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4bd4zz" Jul 08 19:12:16.152975 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:16.152936 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-pnwjc" Jul 08 19:12:16.154656 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:16.154635 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-p2b8c" Jul 08 19:12:23.196492 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:12:23.196463 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6f5b6f7879-lrlh5" Jul 08 19:13:15.036773 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.036729 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x"] Jul 08 19:13:15.039656 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.039630 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x" Jul 08 19:13:15.042228 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.042204 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 19:13:15.044292 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.044265 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 19:13:15.044391 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.044269 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vjhgt\"" Jul 08 19:13:15.049363 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.049342 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x"] Jul 08 19:13:15.073865 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.073841 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gdmg\" (UniqueName: \"kubernetes.io/projected/39fab212-2aa5-419f-acbb-7a96d731cac6-kube-api-access-2gdmg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x\" (UID: \"39fab212-2aa5-419f-acbb-7a96d731cac6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x" Jul 08 19:13:15.073960 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.073941 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/39fab212-2aa5-419f-acbb-7a96d731cac6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x\" (UID: \"39fab212-2aa5-419f-acbb-7a96d731cac6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x" Jul 08 19:13:15.074019 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.074004 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/39fab212-2aa5-419f-acbb-7a96d731cac6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x\" (UID: \"39fab212-2aa5-419f-acbb-7a96d731cac6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x" Jul 08 19:13:15.174394 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.174363 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/39fab212-2aa5-419f-acbb-7a96d731cac6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x\" (UID: \"39fab212-2aa5-419f-acbb-7a96d731cac6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x" Jul 08 19:13:15.174494 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.174415 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gdmg\" (UniqueName: \"kubernetes.io/projected/39fab212-2aa5-419f-acbb-7a96d731cac6-kube-api-access-2gdmg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x\" (UID: \"39fab212-2aa5-419f-acbb-7a96d731cac6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x" Jul 08 19:13:15.174494 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.174460 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/39fab212-2aa5-419f-acbb-7a96d731cac6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x\" (UID: \"39fab212-2aa5-419f-acbb-7a96d731cac6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x" Jul 08 19:13:15.174798 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.174777 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/39fab212-2aa5-419f-acbb-7a96d731cac6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x\" (UID: \"39fab212-2aa5-419f-acbb-7a96d731cac6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x" Jul 08 19:13:15.174860 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.174804 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/39fab212-2aa5-419f-acbb-7a96d731cac6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x\" (UID: \"39fab212-2aa5-419f-acbb-7a96d731cac6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x" Jul 08 19:13:15.184665 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.184645 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gdmg\" (UniqueName: \"kubernetes.io/projected/39fab212-2aa5-419f-acbb-7a96d731cac6-kube-api-access-2gdmg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x\" (UID: \"39fab212-2aa5-419f-acbb-7a96d731cac6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x" Jul 08 19:13:15.349368 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.349315 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x" Jul 08 19:13:15.484557 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:13:15.484504 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39fab212_2aa5_419f_acbb_7a96d731cac6.slice/crio-27f43d2a9cc5d23cb898e296475876a82e766127ee51a2d2c196b45917ed283a WatchSource:0}: Error finding container 27f43d2a9cc5d23cb898e296475876a82e766127ee51a2d2c196b45917ed283a: Status 404 returned error can't find the container with id 27f43d2a9cc5d23cb898e296475876a82e766127ee51a2d2c196b45917ed283a Jul 08 19:13:15.484740 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.484666 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x"] Jul 08 19:13:15.500580 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.500556 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x" event={"ID":"39fab212-2aa5-419f-acbb-7a96d731cac6","Type":"ContainerStarted","Data":"27f43d2a9cc5d23cb898e296475876a82e766127ee51a2d2c196b45917ed283a"} Jul 08 19:13:15.863552 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.863516 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m"] Jul 08 19:13:15.866068 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.866052 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m" Jul 08 19:13:15.894067 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.894043 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m"] Jul 08 19:13:15.980249 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.980227 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg79m\" (UniqueName: \"kubernetes.io/projected/9a154f14-66b0-47d2-a6a9-1339729905ab-kube-api-access-kg79m\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m\" (UID: \"9a154f14-66b0-47d2-a6a9-1339729905ab\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m" Jul 08 19:13:15.980350 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.980274 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a154f14-66b0-47d2-a6a9-1339729905ab-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m\" (UID: \"9a154f14-66b0-47d2-a6a9-1339729905ab\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m" Jul 08 19:13:15.980409 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:15.980389 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a154f14-66b0-47d2-a6a9-1339729905ab-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m\" (UID: \"9a154f14-66b0-47d2-a6a9-1339729905ab\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m" Jul 08 19:13:16.081513 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.081492 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a154f14-66b0-47d2-a6a9-1339729905ab-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m\" (UID: \"9a154f14-66b0-47d2-a6a9-1339729905ab\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m" Jul 08 19:13:16.081817 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.081560 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a154f14-66b0-47d2-a6a9-1339729905ab-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m\" (UID: \"9a154f14-66b0-47d2-a6a9-1339729905ab\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m" Jul 08 19:13:16.081817 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.081593 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg79m\" (UniqueName: \"kubernetes.io/projected/9a154f14-66b0-47d2-a6a9-1339729905ab-kube-api-access-kg79m\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m\" (UID: \"9a154f14-66b0-47d2-a6a9-1339729905ab\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m" Jul 08 19:13:16.081892 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.081840 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a154f14-66b0-47d2-a6a9-1339729905ab-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m\" (UID: \"9a154f14-66b0-47d2-a6a9-1339729905ab\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m" Jul 08 19:13:16.081926 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.081906 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a154f14-66b0-47d2-a6a9-1339729905ab-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m\" (UID: \"9a154f14-66b0-47d2-a6a9-1339729905ab\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m" Jul 08 19:13:16.091361 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.091340 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg79m\" (UniqueName: \"kubernetes.io/projected/9a154f14-66b0-47d2-a6a9-1339729905ab-kube-api-access-kg79m\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m\" (UID: \"9a154f14-66b0-47d2-a6a9-1339729905ab\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m" Jul 08 19:13:16.174696 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.174673 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m" Jul 08 19:13:16.327666 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.327616 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m"] Jul 08 19:13:16.330183 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:13:16.330157 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a154f14_66b0_47d2_a6a9_1339729905ab.slice/crio-d448ff7a5a3c7d701b725811df011a678270e4b9282f6f05df6594acf618f4ec WatchSource:0}: Error finding container d448ff7a5a3c7d701b725811df011a678270e4b9282f6f05df6594acf618f4ec: Status 404 returned error can't find the container with id d448ff7a5a3c7d701b725811df011a678270e4b9282f6f05df6594acf618f4ec Jul 08 19:13:16.478632 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.478604 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd"] Jul 08 19:13:16.481162 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.481147 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd" Jul 08 19:13:16.494935 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.494917 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd"] Jul 08 19:13:16.507828 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.507809 2560 generic.go:356] "Generic (PLEG): container finished" podID="9a154f14-66b0-47d2-a6a9-1339729905ab" containerID="8b1a34bded72d66a6ab5cfebd4504bd4eb23ae1ec43801f3e2b9f3206b524376" exitCode=0 Jul 08 19:13:16.507924 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.507895 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m" event={"ID":"9a154f14-66b0-47d2-a6a9-1339729905ab","Type":"ContainerDied","Data":"8b1a34bded72d66a6ab5cfebd4504bd4eb23ae1ec43801f3e2b9f3206b524376"} Jul 08 19:13:16.507973 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.507935 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m" event={"ID":"9a154f14-66b0-47d2-a6a9-1339729905ab","Type":"ContainerStarted","Data":"d448ff7a5a3c7d701b725811df011a678270e4b9282f6f05df6594acf618f4ec"} Jul 08 19:13:16.509486 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.509462 2560 generic.go:356] "Generic (PLEG): container finished" podID="39fab212-2aa5-419f-acbb-7a96d731cac6" containerID="1a202e8890bf1d756c7cf9c7f0fad2a9f5f5e0c49d60c47ac9d255bddbfd0c54" exitCode=0 Jul 08 19:13:16.509632 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.509508 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x" event={"ID":"39fab212-2aa5-419f-acbb-7a96d731cac6","Type":"ContainerDied","Data":"1a202e8890bf1d756c7cf9c7f0fad2a9f5f5e0c49d60c47ac9d255bddbfd0c54"} Jul 08 19:13:16.588060 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.588037 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj99c\" (UniqueName: \"kubernetes.io/projected/d95cfc77-c5bb-4427-bbd6-d07de14710ae-kube-api-access-wj99c\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd\" (UID: \"d95cfc77-c5bb-4427-bbd6-d07de14710ae\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd" Jul 08 19:13:16.588174 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.588148 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d95cfc77-c5bb-4427-bbd6-d07de14710ae-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd\" (UID: \"d95cfc77-c5bb-4427-bbd6-d07de14710ae\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd" Jul 08 19:13:16.588286 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.588266 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d95cfc77-c5bb-4427-bbd6-d07de14710ae-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd\" (UID: \"d95cfc77-c5bb-4427-bbd6-d07de14710ae\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd" Jul 08 19:13:16.689211 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.689151 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d95cfc77-c5bb-4427-bbd6-d07de14710ae-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd\" (UID: \"d95cfc77-c5bb-4427-bbd6-d07de14710ae\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd" Jul 08 19:13:16.689211 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.689195 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d95cfc77-c5bb-4427-bbd6-d07de14710ae-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd\" (UID: \"d95cfc77-c5bb-4427-bbd6-d07de14710ae\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd" Jul 08 19:13:16.689363 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.689227 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj99c\" (UniqueName: \"kubernetes.io/projected/d95cfc77-c5bb-4427-bbd6-d07de14710ae-kube-api-access-wj99c\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd\" (UID: \"d95cfc77-c5bb-4427-bbd6-d07de14710ae\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd" Jul 08 19:13:16.689564 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.689518 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d95cfc77-c5bb-4427-bbd6-d07de14710ae-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd\" (UID: \"d95cfc77-c5bb-4427-bbd6-d07de14710ae\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd" Jul 08 19:13:16.689645 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.689584 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d95cfc77-c5bb-4427-bbd6-d07de14710ae-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd\" (UID: \"d95cfc77-c5bb-4427-bbd6-d07de14710ae\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd" Jul 08 19:13:16.699633 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.699615 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj99c\" (UniqueName: \"kubernetes.io/projected/d95cfc77-c5bb-4427-bbd6-d07de14710ae-kube-api-access-wj99c\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd\" (UID: \"d95cfc77-c5bb-4427-bbd6-d07de14710ae\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd" Jul 08 19:13:16.799403 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.799380 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd" Jul 08 19:13:16.862475 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.862443 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr"] Jul 08 19:13:16.865209 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.865187 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr" Jul 08 19:13:16.882939 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.882899 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr"] Jul 08 19:13:16.935978 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.935955 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd"] Jul 08 19:13:16.937192 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:13:16.937164 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd95cfc77_c5bb_4427_bbd6_d07de14710ae.slice/crio-c7bb71d8de10d7923a78cad91079820217c5bb0430d65ebb9796aab8ddc725ce WatchSource:0}: Error finding container c7bb71d8de10d7923a78cad91079820217c5bb0430d65ebb9796aab8ddc725ce: Status 404 returned error can't find the container with id c7bb71d8de10d7923a78cad91079820217c5bb0430d65ebb9796aab8ddc725ce Jul 08 19:13:16.999231 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.997072 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np29f\" (UniqueName: \"kubernetes.io/projected/d213b05c-939a-46b5-bcf5-38baab0e3e7b-kube-api-access-np29f\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr\" (UID: \"d213b05c-939a-46b5-bcf5-38baab0e3e7b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr" Jul 08 19:13:16.999231 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.997195 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d213b05c-939a-46b5-bcf5-38baab0e3e7b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr\" (UID: \"d213b05c-939a-46b5-bcf5-38baab0e3e7b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr" Jul 08 19:13:16.999231 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:16.997248 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d213b05c-939a-46b5-bcf5-38baab0e3e7b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr\" (UID: \"d213b05c-939a-46b5-bcf5-38baab0e3e7b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr" Jul 08 19:13:17.098437 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:17.098404 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d213b05c-939a-46b5-bcf5-38baab0e3e7b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr\" (UID: \"d213b05c-939a-46b5-bcf5-38baab0e3e7b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr" Jul 08 19:13:17.098868 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:17.098457 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d213b05c-939a-46b5-bcf5-38baab0e3e7b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr\" (UID: \"d213b05c-939a-46b5-bcf5-38baab0e3e7b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr" Jul 08 19:13:17.098868 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:17.098597 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np29f\" (UniqueName: \"kubernetes.io/projected/d213b05c-939a-46b5-bcf5-38baab0e3e7b-kube-api-access-np29f\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr\" (UID: \"d213b05c-939a-46b5-bcf5-38baab0e3e7b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr" Jul 08 19:13:17.098868 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:17.098856 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d213b05c-939a-46b5-bcf5-38baab0e3e7b-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr\" (UID: \"d213b05c-939a-46b5-bcf5-38baab0e3e7b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr" Jul 08 19:13:17.099029 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:17.098922 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d213b05c-939a-46b5-bcf5-38baab0e3e7b-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr\" (UID: \"d213b05c-939a-46b5-bcf5-38baab0e3e7b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr" Jul 08 19:13:17.111454 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:17.111433 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-np29f\" (UniqueName: \"kubernetes.io/projected/d213b05c-939a-46b5-bcf5-38baab0e3e7b-kube-api-access-np29f\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr\" (UID: \"d213b05c-939a-46b5-bcf5-38baab0e3e7b\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr" Jul 08 19:13:17.174647 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:17.174615 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr" Jul 08 19:13:17.393215 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:17.393186 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr"] Jul 08 19:13:17.495262 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:13:17.495203 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd213b05c_939a_46b5_bcf5_38baab0e3e7b.slice/crio-1528eb76f836eb4cc0587d44f7b6e2bc1178ae29b7732855d858e49d61be85a5 WatchSource:0}: Error finding container 1528eb76f836eb4cc0587d44f7b6e2bc1178ae29b7732855d858e49d61be85a5: Status 404 returned error can't find the container with id 1528eb76f836eb4cc0587d44f7b6e2bc1178ae29b7732855d858e49d61be85a5 Jul 08 19:13:17.517344 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:17.517318 2560 generic.go:356] "Generic (PLEG): container finished" podID="9a154f14-66b0-47d2-a6a9-1339729905ab" containerID="86f965ff65aa49a8f14cfc8b6eaa2fbee69f5f5100b28bcfd5704c32564b659b" exitCode=0 Jul 08 19:13:17.517438 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:17.517407 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m" event={"ID":"9a154f14-66b0-47d2-a6a9-1339729905ab","Type":"ContainerDied","Data":"86f965ff65aa49a8f14cfc8b6eaa2fbee69f5f5100b28bcfd5704c32564b659b"} Jul 08 19:13:17.518691 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:17.518666 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr" event={"ID":"d213b05c-939a-46b5-bcf5-38baab0e3e7b","Type":"ContainerStarted","Data":"1528eb76f836eb4cc0587d44f7b6e2bc1178ae29b7732855d858e49d61be85a5"} Jul 08 19:13:17.520118 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:17.520072 2560 generic.go:356] "Generic (PLEG): container finished" podID="d95cfc77-c5bb-4427-bbd6-d07de14710ae" containerID="22d2cac5c914ce3feac265b4e24e7c92a3c7f60c29364bf1568a9e0a6365f18d" exitCode=0 Jul 08 19:13:17.520118 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:17.520106 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd" event={"ID":"d95cfc77-c5bb-4427-bbd6-d07de14710ae","Type":"ContainerDied","Data":"22d2cac5c914ce3feac265b4e24e7c92a3c7f60c29364bf1568a9e0a6365f18d"} Jul 08 19:13:17.520241 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:17.520145 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd" event={"ID":"d95cfc77-c5bb-4427-bbd6-d07de14710ae","Type":"ContainerStarted","Data":"c7bb71d8de10d7923a78cad91079820217c5bb0430d65ebb9796aab8ddc725ce"} Jul 08 19:13:17.523125 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:17.523100 2560 generic.go:356] "Generic (PLEG): container finished" podID="39fab212-2aa5-419f-acbb-7a96d731cac6" containerID="8f3013c1e904390627590abee91f02cd1f8ccb96d997fd32e51ccb17756aa13c" exitCode=0 Jul 08 19:13:17.523237 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:17.523150 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x" event={"ID":"39fab212-2aa5-419f-acbb-7a96d731cac6","Type":"ContainerDied","Data":"8f3013c1e904390627590abee91f02cd1f8ccb96d997fd32e51ccb17756aa13c"} Jul 08 19:13:18.529839 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:18.529779 2560 generic.go:356] "Generic (PLEG): container finished" podID="d213b05c-939a-46b5-bcf5-38baab0e3e7b" containerID="0cf8b6353625173578ce054fe42a53d794ac0f26821fcc0c5d3017e4ecf5a7be" exitCode=0 Jul 08 19:13:18.530157 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:18.529853 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr" event={"ID":"d213b05c-939a-46b5-bcf5-38baab0e3e7b","Type":"ContainerDied","Data":"0cf8b6353625173578ce054fe42a53d794ac0f26821fcc0c5d3017e4ecf5a7be"} Jul 08 19:13:18.531808 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:18.531781 2560 generic.go:356] "Generic (PLEG): container finished" podID="d95cfc77-c5bb-4427-bbd6-d07de14710ae" containerID="e33805374098aee278f8908590ae3fdddb9be68601b42288fbde8a699decfae6" exitCode=0 Jul 08 19:13:18.531898 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:18.531873 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd" event={"ID":"d95cfc77-c5bb-4427-bbd6-d07de14710ae","Type":"ContainerDied","Data":"e33805374098aee278f8908590ae3fdddb9be68601b42288fbde8a699decfae6"} Jul 08 19:13:18.534214 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:18.534190 2560 generic.go:356] "Generic (PLEG): container finished" podID="39fab212-2aa5-419f-acbb-7a96d731cac6" containerID="bba82d6a350d12c74e2a098d10331507d8f17fa33f8771d0cfca2a068d2305ab" exitCode=0 Jul 08 19:13:18.534299 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:18.534265 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x" event={"ID":"39fab212-2aa5-419f-acbb-7a96d731cac6","Type":"ContainerDied","Data":"bba82d6a350d12c74e2a098d10331507d8f17fa33f8771d0cfca2a068d2305ab"} Jul 08 19:13:18.536406 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:18.536380 2560 generic.go:356] "Generic (PLEG): container finished" podID="9a154f14-66b0-47d2-a6a9-1339729905ab" containerID="1d2cd56b6e3c18d6a7764295edc07c022efa10a8d5e0f45b8f9997f164452299" exitCode=0 Jul 08 19:13:18.536465 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:18.536443 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m" event={"ID":"9a154f14-66b0-47d2-a6a9-1339729905ab","Type":"ContainerDied","Data":"1d2cd56b6e3c18d6a7764295edc07c022efa10a8d5e0f45b8f9997f164452299"} Jul 08 19:13:19.543857 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.543786 2560 generic.go:356] "Generic (PLEG): container finished" podID="d213b05c-939a-46b5-bcf5-38baab0e3e7b" containerID="96f516619e8afc679832afde85e777a2676139ccb571f7c9d6257fa476813fef" exitCode=0 Jul 08 19:13:19.544241 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.543872 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr" event={"ID":"d213b05c-939a-46b5-bcf5-38baab0e3e7b","Type":"ContainerDied","Data":"96f516619e8afc679832afde85e777a2676139ccb571f7c9d6257fa476813fef"} Jul 08 19:13:19.546128 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.546110 2560 generic.go:356] "Generic (PLEG): container finished" podID="d95cfc77-c5bb-4427-bbd6-d07de14710ae" containerID="8918b96a2d0e7a8bba3171ac667c0a3c63f31e740024b73a22c2d37412b7eb22" exitCode=0 Jul 08 19:13:19.546220 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.546197 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd" event={"ID":"d95cfc77-c5bb-4427-bbd6-d07de14710ae","Type":"ContainerDied","Data":"8918b96a2d0e7a8bba3171ac667c0a3c63f31e740024b73a22c2d37412b7eb22"} Jul 08 19:13:19.712287 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.712267 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x" Jul 08 19:13:19.715580 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.715561 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m" Jul 08 19:13:19.819867 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.819804 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a154f14-66b0-47d2-a6a9-1339729905ab-util\") pod \"9a154f14-66b0-47d2-a6a9-1339729905ab\" (UID: \"9a154f14-66b0-47d2-a6a9-1339729905ab\") " Jul 08 19:13:19.819867 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.819843 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/39fab212-2aa5-419f-acbb-7a96d731cac6-bundle\") pod \"39fab212-2aa5-419f-acbb-7a96d731cac6\" (UID: \"39fab212-2aa5-419f-acbb-7a96d731cac6\") " Jul 08 19:13:19.819867 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.819867 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2gdmg\" (UniqueName: \"kubernetes.io/projected/39fab212-2aa5-419f-acbb-7a96d731cac6-kube-api-access-2gdmg\") pod \"39fab212-2aa5-419f-acbb-7a96d731cac6\" (UID: \"39fab212-2aa5-419f-acbb-7a96d731cac6\") " Jul 08 19:13:19.820100 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.819921 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kg79m\" (UniqueName: \"kubernetes.io/projected/9a154f14-66b0-47d2-a6a9-1339729905ab-kube-api-access-kg79m\") pod \"9a154f14-66b0-47d2-a6a9-1339729905ab\" (UID: \"9a154f14-66b0-47d2-a6a9-1339729905ab\") " Jul 08 19:13:19.820100 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.819970 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a154f14-66b0-47d2-a6a9-1339729905ab-bundle\") pod \"9a154f14-66b0-47d2-a6a9-1339729905ab\" (UID: \"9a154f14-66b0-47d2-a6a9-1339729905ab\") " Jul 08 19:13:19.820100 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.820019 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/39fab212-2aa5-419f-acbb-7a96d731cac6-util\") pod \"39fab212-2aa5-419f-acbb-7a96d731cac6\" (UID: \"39fab212-2aa5-419f-acbb-7a96d731cac6\") " Jul 08 19:13:19.820436 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.820410 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39fab212-2aa5-419f-acbb-7a96d731cac6-bundle" (OuterVolumeSpecName: "bundle") pod "39fab212-2aa5-419f-acbb-7a96d731cac6" (UID: "39fab212-2aa5-419f-acbb-7a96d731cac6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:13:19.820610 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.820565 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a154f14-66b0-47d2-a6a9-1339729905ab-bundle" (OuterVolumeSpecName: "bundle") pod "9a154f14-66b0-47d2-a6a9-1339729905ab" (UID: "9a154f14-66b0-47d2-a6a9-1339729905ab"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:13:19.822212 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.822191 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a154f14-66b0-47d2-a6a9-1339729905ab-kube-api-access-kg79m" (OuterVolumeSpecName: "kube-api-access-kg79m") pod "9a154f14-66b0-47d2-a6a9-1339729905ab" (UID: "9a154f14-66b0-47d2-a6a9-1339729905ab"). InnerVolumeSpecName "kube-api-access-kg79m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:13:19.822460 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.822440 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39fab212-2aa5-419f-acbb-7a96d731cac6-kube-api-access-2gdmg" (OuterVolumeSpecName: "kube-api-access-2gdmg") pod "39fab212-2aa5-419f-acbb-7a96d731cac6" (UID: "39fab212-2aa5-419f-acbb-7a96d731cac6"). InnerVolumeSpecName "kube-api-access-2gdmg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:13:19.825917 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.825887 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39fab212-2aa5-419f-acbb-7a96d731cac6-util" (OuterVolumeSpecName: "util") pod "39fab212-2aa5-419f-acbb-7a96d731cac6" (UID: "39fab212-2aa5-419f-acbb-7a96d731cac6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:13:19.826421 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.826403 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a154f14-66b0-47d2-a6a9-1339729905ab-util" (OuterVolumeSpecName: "util") pod "9a154f14-66b0-47d2-a6a9-1339729905ab" (UID: "9a154f14-66b0-47d2-a6a9-1339729905ab"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:13:19.921492 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.921467 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2gdmg\" (UniqueName: \"kubernetes.io/projected/39fab212-2aa5-419f-acbb-7a96d731cac6-kube-api-access-2gdmg\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:13:19.921492 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.921487 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kg79m\" (UniqueName: \"kubernetes.io/projected/9a154f14-66b0-47d2-a6a9-1339729905ab-kube-api-access-kg79m\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:13:19.921492 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.921497 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a154f14-66b0-47d2-a6a9-1339729905ab-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:13:19.921682 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.921505 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/39fab212-2aa5-419f-acbb-7a96d731cac6-util\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:13:19.921682 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.921513 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a154f14-66b0-47d2-a6a9-1339729905ab-util\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:13:19.921682 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:19.921520 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/39fab212-2aa5-419f-acbb-7a96d731cac6-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:13:20.552700 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:20.552646 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x" Jul 08 19:13:20.553041 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:20.552642 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x" event={"ID":"39fab212-2aa5-419f-acbb-7a96d731cac6","Type":"ContainerDied","Data":"27f43d2a9cc5d23cb898e296475876a82e766127ee51a2d2c196b45917ed283a"} Jul 08 19:13:20.553041 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:20.552808 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="27f43d2a9cc5d23cb898e296475876a82e766127ee51a2d2c196b45917ed283a" Jul 08 19:13:20.554872 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:20.554852 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m" Jul 08 19:13:20.554986 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:20.554867 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m" event={"ID":"9a154f14-66b0-47d2-a6a9-1339729905ab","Type":"ContainerDied","Data":"d448ff7a5a3c7d701b725811df011a678270e4b9282f6f05df6594acf618f4ec"} Jul 08 19:13:20.554986 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:20.554909 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d448ff7a5a3c7d701b725811df011a678270e4b9282f6f05df6594acf618f4ec" Jul 08 19:13:20.557343 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:20.557316 2560 generic.go:356] "Generic (PLEG): container finished" podID="d213b05c-939a-46b5-bcf5-38baab0e3e7b" containerID="ef052664c0b8870d3e5ab26f38c056be23ae2892baeb2655644f15d4b2fe56f7" exitCode=0 Jul 08 19:13:20.557458 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:20.557390 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr" event={"ID":"d213b05c-939a-46b5-bcf5-38baab0e3e7b","Type":"ContainerDied","Data":"ef052664c0b8870d3e5ab26f38c056be23ae2892baeb2655644f15d4b2fe56f7"} Jul 08 19:13:20.683154 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:20.683128 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd" Jul 08 19:13:20.727747 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:20.727727 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wj99c\" (UniqueName: \"kubernetes.io/projected/d95cfc77-c5bb-4427-bbd6-d07de14710ae-kube-api-access-wj99c\") pod \"d95cfc77-c5bb-4427-bbd6-d07de14710ae\" (UID: \"d95cfc77-c5bb-4427-bbd6-d07de14710ae\") " Jul 08 19:13:20.727843 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:20.727777 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d95cfc77-c5bb-4427-bbd6-d07de14710ae-util\") pod \"d95cfc77-c5bb-4427-bbd6-d07de14710ae\" (UID: \"d95cfc77-c5bb-4427-bbd6-d07de14710ae\") " Jul 08 19:13:20.727843 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:20.727815 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d95cfc77-c5bb-4427-bbd6-d07de14710ae-bundle\") pod \"d95cfc77-c5bb-4427-bbd6-d07de14710ae\" (UID: \"d95cfc77-c5bb-4427-bbd6-d07de14710ae\") " Jul 08 19:13:20.728497 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:20.728453 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d95cfc77-c5bb-4427-bbd6-d07de14710ae-bundle" (OuterVolumeSpecName: "bundle") pod "d95cfc77-c5bb-4427-bbd6-d07de14710ae" (UID: "d95cfc77-c5bb-4427-bbd6-d07de14710ae"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:13:20.729919 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:20.729892 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d95cfc77-c5bb-4427-bbd6-d07de14710ae-kube-api-access-wj99c" (OuterVolumeSpecName: "kube-api-access-wj99c") pod "d95cfc77-c5bb-4427-bbd6-d07de14710ae" (UID: "d95cfc77-c5bb-4427-bbd6-d07de14710ae"). InnerVolumeSpecName "kube-api-access-wj99c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:13:20.732958 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:20.732939 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d95cfc77-c5bb-4427-bbd6-d07de14710ae-util" (OuterVolumeSpecName: "util") pod "d95cfc77-c5bb-4427-bbd6-d07de14710ae" (UID: "d95cfc77-c5bb-4427-bbd6-d07de14710ae"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:13:20.829250 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:20.829197 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d95cfc77-c5bb-4427-bbd6-d07de14710ae-util\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:13:20.829250 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:20.829218 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d95cfc77-c5bb-4427-bbd6-d07de14710ae-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:13:20.829250 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:20.829227 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wj99c\" (UniqueName: \"kubernetes.io/projected/d95cfc77-c5bb-4427-bbd6-d07de14710ae-kube-api-access-wj99c\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:13:21.565588 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:21.565560 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd" Jul 08 19:13:21.565588 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:21.565567 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd" event={"ID":"d95cfc77-c5bb-4427-bbd6-d07de14710ae","Type":"ContainerDied","Data":"c7bb71d8de10d7923a78cad91079820217c5bb0430d65ebb9796aab8ddc725ce"} Jul 08 19:13:21.566089 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:21.565625 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c7bb71d8de10d7923a78cad91079820217c5bb0430d65ebb9796aab8ddc725ce" Jul 08 19:13:21.694202 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:21.694180 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr" Jul 08 19:13:21.837151 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:21.837094 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d213b05c-939a-46b5-bcf5-38baab0e3e7b-bundle\") pod \"d213b05c-939a-46b5-bcf5-38baab0e3e7b\" (UID: \"d213b05c-939a-46b5-bcf5-38baab0e3e7b\") " Jul 08 19:13:21.837151 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:21.837131 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d213b05c-939a-46b5-bcf5-38baab0e3e7b-util\") pod \"d213b05c-939a-46b5-bcf5-38baab0e3e7b\" (UID: \"d213b05c-939a-46b5-bcf5-38baab0e3e7b\") " Jul 08 19:13:21.837334 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:21.837176 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-np29f\" (UniqueName: \"kubernetes.io/projected/d213b05c-939a-46b5-bcf5-38baab0e3e7b-kube-api-access-np29f\") pod \"d213b05c-939a-46b5-bcf5-38baab0e3e7b\" (UID: \"d213b05c-939a-46b5-bcf5-38baab0e3e7b\") " Jul 08 19:13:21.837720 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:21.837689 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d213b05c-939a-46b5-bcf5-38baab0e3e7b-bundle" (OuterVolumeSpecName: "bundle") pod "d213b05c-939a-46b5-bcf5-38baab0e3e7b" (UID: "d213b05c-939a-46b5-bcf5-38baab0e3e7b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:13:21.839421 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:21.839390 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d213b05c-939a-46b5-bcf5-38baab0e3e7b-kube-api-access-np29f" (OuterVolumeSpecName: "kube-api-access-np29f") pod "d213b05c-939a-46b5-bcf5-38baab0e3e7b" (UID: "d213b05c-939a-46b5-bcf5-38baab0e3e7b"). InnerVolumeSpecName "kube-api-access-np29f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:13:21.842596 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:21.842562 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d213b05c-939a-46b5-bcf5-38baab0e3e7b-util" (OuterVolumeSpecName: "util") pod "d213b05c-939a-46b5-bcf5-38baab0e3e7b" (UID: "d213b05c-939a-46b5-bcf5-38baab0e3e7b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 19:13:21.938189 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:21.938167 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d213b05c-939a-46b5-bcf5-38baab0e3e7b-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:13:21.938189 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:21.938187 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d213b05c-939a-46b5-bcf5-38baab0e3e7b-util\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:13:21.938309 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:21.938196 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-np29f\" (UniqueName: \"kubernetes.io/projected/d213b05c-939a-46b5-bcf5-38baab0e3e7b-kube-api-access-np29f\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:13:22.573254 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:22.573221 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr" Jul 08 19:13:22.573254 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:22.573234 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr" event={"ID":"d213b05c-939a-46b5-bcf5-38baab0e3e7b","Type":"ContainerDied","Data":"1528eb76f836eb4cc0587d44f7b6e2bc1178ae29b7732855d858e49d61be85a5"} Jul 08 19:13:22.573672 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:22.573279 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1528eb76f836eb4cc0587d44f7b6e2bc1178ae29b7732855d858e49d61be85a5" Jul 08 19:13:31.023427 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:31.023392 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d596bd57c-mtj8c"] Jul 08 19:13:46.302692 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.302660 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-5tp4x"] Jul 08 19:13:46.303127 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303024 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39fab212-2aa5-419f-acbb-7a96d731cac6" containerName="util" Jul 08 19:13:46.303127 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303034 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="39fab212-2aa5-419f-acbb-7a96d731cac6" containerName="util" Jul 08 19:13:46.303127 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303041 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a154f14-66b0-47d2-a6a9-1339729905ab" containerName="util" Jul 08 19:13:46.303127 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303047 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a154f14-66b0-47d2-a6a9-1339729905ab" containerName="util" Jul 08 19:13:46.303127 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303053 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d95cfc77-c5bb-4427-bbd6-d07de14710ae" containerName="util" Jul 08 19:13:46.303127 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303058 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d95cfc77-c5bb-4427-bbd6-d07de14710ae" containerName="util" Jul 08 19:13:46.303127 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303067 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d213b05c-939a-46b5-bcf5-38baab0e3e7b" containerName="extract" Jul 08 19:13:46.303127 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303072 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d213b05c-939a-46b5-bcf5-38baab0e3e7b" containerName="extract" Jul 08 19:13:46.303127 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303081 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39fab212-2aa5-419f-acbb-7a96d731cac6" containerName="extract" Jul 08 19:13:46.303127 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303086 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="39fab212-2aa5-419f-acbb-7a96d731cac6" containerName="extract" Jul 08 19:13:46.303127 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303091 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d213b05c-939a-46b5-bcf5-38baab0e3e7b" containerName="pull" Jul 08 19:13:46.303127 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303096 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d213b05c-939a-46b5-bcf5-38baab0e3e7b" containerName="pull" Jul 08 19:13:46.303127 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303105 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d213b05c-939a-46b5-bcf5-38baab0e3e7b" containerName="util" Jul 08 19:13:46.303127 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303109 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d213b05c-939a-46b5-bcf5-38baab0e3e7b" containerName="util" Jul 08 19:13:46.303127 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303123 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a154f14-66b0-47d2-a6a9-1339729905ab" containerName="extract" Jul 08 19:13:46.303127 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303129 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a154f14-66b0-47d2-a6a9-1339729905ab" containerName="extract" Jul 08 19:13:46.303127 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303134 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d95cfc77-c5bb-4427-bbd6-d07de14710ae" containerName="pull" Jul 08 19:13:46.303647 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303139 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d95cfc77-c5bb-4427-bbd6-d07de14710ae" containerName="pull" Jul 08 19:13:46.303647 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303145 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a154f14-66b0-47d2-a6a9-1339729905ab" containerName="pull" Jul 08 19:13:46.303647 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303149 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a154f14-66b0-47d2-a6a9-1339729905ab" containerName="pull" Jul 08 19:13:46.303647 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303155 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d95cfc77-c5bb-4427-bbd6-d07de14710ae" containerName="extract" Jul 08 19:13:46.303647 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303159 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d95cfc77-c5bb-4427-bbd6-d07de14710ae" containerName="extract" Jul 08 19:13:46.303647 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303167 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39fab212-2aa5-419f-acbb-7a96d731cac6" containerName="pull" Jul 08 19:13:46.303647 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303172 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="39fab212-2aa5-419f-acbb-7a96d731cac6" containerName="pull" Jul 08 19:13:46.303647 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303219 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d95cfc77-c5bb-4427-bbd6-d07de14710ae" containerName="extract" Jul 08 19:13:46.303647 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303228 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="39fab212-2aa5-419f-acbb-7a96d731cac6" containerName="extract" Jul 08 19:13:46.303647 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303233 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d213b05c-939a-46b5-bcf5-38baab0e3e7b" containerName="extract" Jul 08 19:13:46.303647 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.303238 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a154f14-66b0-47d2-a6a9-1339729905ab" containerName="extract" Jul 08 19:13:46.305795 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.305779 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5tp4x" Jul 08 19:13:46.308313 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.308294 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 08 19:13:46.308313 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.308303 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 19:13:46.308479 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.308347 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 08 19:13:46.309087 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.309070 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vjhgt\"" Jul 08 19:13:46.309138 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.309070 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 19:13:46.315645 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.315625 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-5tp4x"] Jul 08 19:13:46.421373 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.421340 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/226ade13-1862-4b6f-a4f7-e24ea6fb44ad-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5tp4x\" (UID: \"226ade13-1862-4b6f-a4f7-e24ea6fb44ad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5tp4x" Jul 08 19:13:46.421628 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.421492 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/226ade13-1862-4b6f-a4f7-e24ea6fb44ad-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-5tp4x\" (UID: \"226ade13-1862-4b6f-a4f7-e24ea6fb44ad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5tp4x" Jul 08 19:13:46.421628 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.421520 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk2w6\" (UniqueName: \"kubernetes.io/projected/226ade13-1862-4b6f-a4f7-e24ea6fb44ad-kube-api-access-bk2w6\") pod \"kuadrant-console-plugin-66b5774985-5tp4x\" (UID: \"226ade13-1862-4b6f-a4f7-e24ea6fb44ad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5tp4x" Jul 08 19:13:46.521991 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.521960 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/226ade13-1862-4b6f-a4f7-e24ea6fb44ad-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5tp4x\" (UID: \"226ade13-1862-4b6f-a4f7-e24ea6fb44ad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5tp4x" Jul 08 19:13:46.522129 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.522047 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/226ade13-1862-4b6f-a4f7-e24ea6fb44ad-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-5tp4x\" (UID: \"226ade13-1862-4b6f-a4f7-e24ea6fb44ad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5tp4x" Jul 08 19:13:46.522129 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.522073 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk2w6\" (UniqueName: \"kubernetes.io/projected/226ade13-1862-4b6f-a4f7-e24ea6fb44ad-kube-api-access-bk2w6\") pod \"kuadrant-console-plugin-66b5774985-5tp4x\" (UID: \"226ade13-1862-4b6f-a4f7-e24ea6fb44ad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5tp4x" Jul 08 19:13:46.522129 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:13:46.522118 2560 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 08 19:13:46.522251 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:13:46.522193 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/226ade13-1862-4b6f-a4f7-e24ea6fb44ad-plugin-serving-cert podName:226ade13-1862-4b6f-a4f7-e24ea6fb44ad nodeName:}" failed. No retries permitted until 2026-07-08 19:13:47.022175277 +0000 UTC m=+501.200309261 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/226ade13-1862-4b6f-a4f7-e24ea6fb44ad-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-5tp4x" (UID: "226ade13-1862-4b6f-a4f7-e24ea6fb44ad") : secret "plugin-serving-cert" not found Jul 08 19:13:46.522723 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.522704 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/226ade13-1862-4b6f-a4f7-e24ea6fb44ad-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-5tp4x\" (UID: \"226ade13-1862-4b6f-a4f7-e24ea6fb44ad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5tp4x" Jul 08 19:13:46.537969 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:46.537949 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk2w6\" (UniqueName: \"kubernetes.io/projected/226ade13-1862-4b6f-a4f7-e24ea6fb44ad-kube-api-access-bk2w6\") pod \"kuadrant-console-plugin-66b5774985-5tp4x\" (UID: \"226ade13-1862-4b6f-a4f7-e24ea6fb44ad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5tp4x" Jul 08 19:13:47.025865 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:47.025835 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/226ade13-1862-4b6f-a4f7-e24ea6fb44ad-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5tp4x\" (UID: \"226ade13-1862-4b6f-a4f7-e24ea6fb44ad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5tp4x" Jul 08 19:13:47.028366 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:47.028338 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/226ade13-1862-4b6f-a4f7-e24ea6fb44ad-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-5tp4x\" (UID: \"226ade13-1862-4b6f-a4f7-e24ea6fb44ad\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5tp4x" Jul 08 19:13:47.215123 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:47.215090 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5tp4x" Jul 08 19:13:47.360371 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:47.360339 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-5tp4x"] Jul 08 19:13:47.363354 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:13:47.363326 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod226ade13_1862_4b6f_a4f7_e24ea6fb44ad.slice/crio-37c5fd9e458b102956cbe90e2e91d859438d772d0c63d014d0d9ef831fc5231f WatchSource:0}: Error finding container 37c5fd9e458b102956cbe90e2e91d859438d772d0c63d014d0d9ef831fc5231f: Status 404 returned error can't find the container with id 37c5fd9e458b102956cbe90e2e91d859438d772d0c63d014d0d9ef831fc5231f Jul 08 19:13:47.705641 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:47.705599 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5tp4x" event={"ID":"226ade13-1862-4b6f-a4f7-e24ea6fb44ad","Type":"ContainerStarted","Data":"37c5fd9e458b102956cbe90e2e91d859438d772d0c63d014d0d9ef831fc5231f"} Jul 08 19:13:56.050162 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.050114 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-d596bd57c-mtj8c" podUID="cb1c7709-e00a-45fb-b740-48fbba056675" containerName="console" containerID="cri-o://611605f551057392ff3af1fa9ac1497cc6829031528dc7e42a05f80dab5aa0d5" gracePeriod=15 Jul 08 19:13:56.290066 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.290035 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:13:56.417074 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.417040 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-service-ca\") pod \"cb1c7709-e00a-45fb-b740-48fbba056675\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " Jul 08 19:13:56.417233 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.417090 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-oauth-serving-cert\") pod \"cb1c7709-e00a-45fb-b740-48fbba056675\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " Jul 08 19:13:56.417233 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.417131 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb1c7709-e00a-45fb-b740-48fbba056675-console-serving-cert\") pod \"cb1c7709-e00a-45fb-b740-48fbba056675\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " Jul 08 19:13:56.417233 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.417152 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-trusted-ca-bundle\") pod \"cb1c7709-e00a-45fb-b740-48fbba056675\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " Jul 08 19:13:56.417233 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.417170 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-console-config\") pod \"cb1c7709-e00a-45fb-b740-48fbba056675\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " Jul 08 19:13:56.417441 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.417245 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb1c7709-e00a-45fb-b740-48fbba056675-console-oauth-config\") pod \"cb1c7709-e00a-45fb-b740-48fbba056675\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " Jul 08 19:13:56.417441 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.417273 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9j6s4\" (UniqueName: \"kubernetes.io/projected/cb1c7709-e00a-45fb-b740-48fbba056675-kube-api-access-9j6s4\") pod \"cb1c7709-e00a-45fb-b740-48fbba056675\" (UID: \"cb1c7709-e00a-45fb-b740-48fbba056675\") " Jul 08 19:13:56.417859 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.417817 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cb1c7709-e00a-45fb-b740-48fbba056675" (UID: "cb1c7709-e00a-45fb-b740-48fbba056675"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:13:56.417986 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.417854 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cb1c7709-e00a-45fb-b740-48fbba056675" (UID: "cb1c7709-e00a-45fb-b740-48fbba056675"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:13:56.417986 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.417909 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-service-ca" (OuterVolumeSpecName: "service-ca") pod "cb1c7709-e00a-45fb-b740-48fbba056675" (UID: "cb1c7709-e00a-45fb-b740-48fbba056675"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:13:56.417986 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.417954 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-console-config" (OuterVolumeSpecName: "console-config") pod "cb1c7709-e00a-45fb-b740-48fbba056675" (UID: "cb1c7709-e00a-45fb-b740-48fbba056675"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 19:13:56.420003 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.419961 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb1c7709-e00a-45fb-b740-48fbba056675-kube-api-access-9j6s4" (OuterVolumeSpecName: "kube-api-access-9j6s4") pod "cb1c7709-e00a-45fb-b740-48fbba056675" (UID: "cb1c7709-e00a-45fb-b740-48fbba056675"). InnerVolumeSpecName "kube-api-access-9j6s4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:13:56.420003 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.419955 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb1c7709-e00a-45fb-b740-48fbba056675-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cb1c7709-e00a-45fb-b740-48fbba056675" (UID: "cb1c7709-e00a-45fb-b740-48fbba056675"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:13:56.420003 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.419988 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb1c7709-e00a-45fb-b740-48fbba056675-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cb1c7709-e00a-45fb-b740-48fbba056675" (UID: "cb1c7709-e00a-45fb-b740-48fbba056675"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 19:13:56.518743 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.518721 2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb1c7709-e00a-45fb-b740-48fbba056675-console-oauth-config\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:13:56.518743 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.518743 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9j6s4\" (UniqueName: \"kubernetes.io/projected/cb1c7709-e00a-45fb-b740-48fbba056675-kube-api-access-9j6s4\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:13:56.518896 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.518754 2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-service-ca\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:13:56.518896 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.518762 2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-oauth-serving-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:13:56.518896 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.518770 2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb1c7709-e00a-45fb-b740-48fbba056675-console-serving-cert\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:13:56.518896 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.518783 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-trusted-ca-bundle\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:13:56.518896 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.518796 2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb1c7709-e00a-45fb-b740-48fbba056675-console-config\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:13:56.759499 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.759411 2560 generic.go:356] "Generic (PLEG): container finished" podID="cb1c7709-e00a-45fb-b740-48fbba056675" containerID="611605f551057392ff3af1fa9ac1497cc6829031528dc7e42a05f80dab5aa0d5" exitCode=0 Jul 08 19:13:56.759499 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.759478 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d596bd57c-mtj8c" Jul 08 19:13:56.759755 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.759494 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d596bd57c-mtj8c" event={"ID":"cb1c7709-e00a-45fb-b740-48fbba056675","Type":"ContainerDied","Data":"611605f551057392ff3af1fa9ac1497cc6829031528dc7e42a05f80dab5aa0d5"} Jul 08 19:13:56.759755 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.759573 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d596bd57c-mtj8c" event={"ID":"cb1c7709-e00a-45fb-b740-48fbba056675","Type":"ContainerDied","Data":"ca914be693b6f1eafd2d503c50c00c3d2c04bc076e88b9e9283c90937301cf2b"} Jul 08 19:13:56.759755 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.759601 2560 scope.go:117] "RemoveContainer" containerID="611605f551057392ff3af1fa9ac1497cc6829031528dc7e42a05f80dab5aa0d5" Jul 08 19:13:56.769653 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.769634 2560 scope.go:117] "RemoveContainer" containerID="611605f551057392ff3af1fa9ac1497cc6829031528dc7e42a05f80dab5aa0d5" Jul 08 19:13:56.769923 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:13:56.769906 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"611605f551057392ff3af1fa9ac1497cc6829031528dc7e42a05f80dab5aa0d5\": container with ID starting with 611605f551057392ff3af1fa9ac1497cc6829031528dc7e42a05f80dab5aa0d5 not found: ID does not exist" containerID="611605f551057392ff3af1fa9ac1497cc6829031528dc7e42a05f80dab5aa0d5" Jul 08 19:13:56.769960 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.769933 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"611605f551057392ff3af1fa9ac1497cc6829031528dc7e42a05f80dab5aa0d5"} err="failed to get container status \"611605f551057392ff3af1fa9ac1497cc6829031528dc7e42a05f80dab5aa0d5\": rpc error: code = NotFound desc = could not find container \"611605f551057392ff3af1fa9ac1497cc6829031528dc7e42a05f80dab5aa0d5\": container with ID starting with 611605f551057392ff3af1fa9ac1497cc6829031528dc7e42a05f80dab5aa0d5 not found: ID does not exist" Jul 08 19:13:56.797845 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.797826 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d596bd57c-mtj8c"] Jul 08 19:13:56.808016 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:56.807993 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d596bd57c-mtj8c"] Jul 08 19:13:58.355263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:13:58.355231 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb1c7709-e00a-45fb-b740-48fbba056675" path="/var/lib/kubelet/pods/cb1c7709-e00a-45fb-b740-48fbba056675/volumes" Jul 08 19:14:11.851820 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:11.851775 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5tp4x" event={"ID":"226ade13-1862-4b6f-a4f7-e24ea6fb44ad","Type":"ContainerStarted","Data":"c3d883ec092fee2df75b2992d6cd42c2ce47855d6efd78a128ed5d4d79480e40"} Jul 08 19:14:30.862372 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:30.862315 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-5tp4x" podStartSLOduration=20.879512948 podStartE2EDuration="44.86229978s" totalImagesPullingTime="23.982786832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:13:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:14:11.87815335 +0000 UTC m=+526.056287360" watchObservedRunningTime="2026-07-08 19:14:30.86229978 +0000 UTC m=+545.040433785" Jul 08 19:14:30.862809 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:30.862541 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:14:30.862887 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:30.862875 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb1c7709-e00a-45fb-b740-48fbba056675" containerName="console" Jul 08 19:14:30.862940 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:30.862889 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb1c7709-e00a-45fb-b740-48fbba056675" containerName="console" Jul 08 19:14:30.862975 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:30.862952 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb1c7709-e00a-45fb-b740-48fbba056675" containerName="console" Jul 08 19:14:30.928772 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:30.928732 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:14:30.928772 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:30.928775 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:14:30.928958 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:30.928870 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-672sq" Jul 08 19:14:30.931556 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:30.931517 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 08 19:14:31.023715 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:31.023673 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzbwn\" (UniqueName: \"kubernetes.io/projected/1dc80bb7-afa4-438c-86a6-4360003b2ab1-kube-api-access-fzbwn\") pod \"limitador-limitador-54d6d988d5-672sq\" (UID: \"1dc80bb7-afa4-438c-86a6-4360003b2ab1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-672sq" Jul 08 19:14:31.023866 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:31.023780 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1dc80bb7-afa4-438c-86a6-4360003b2ab1-config-file\") pod \"limitador-limitador-54d6d988d5-672sq\" (UID: \"1dc80bb7-afa4-438c-86a6-4360003b2ab1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-672sq" Jul 08 19:14:31.124406 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:31.124326 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzbwn\" (UniqueName: \"kubernetes.io/projected/1dc80bb7-afa4-438c-86a6-4360003b2ab1-kube-api-access-fzbwn\") pod \"limitador-limitador-54d6d988d5-672sq\" (UID: \"1dc80bb7-afa4-438c-86a6-4360003b2ab1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-672sq" Jul 08 19:14:31.124406 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:31.124393 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1dc80bb7-afa4-438c-86a6-4360003b2ab1-config-file\") pod \"limitador-limitador-54d6d988d5-672sq\" (UID: \"1dc80bb7-afa4-438c-86a6-4360003b2ab1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-672sq" Jul 08 19:14:31.124980 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:31.124961 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1dc80bb7-afa4-438c-86a6-4360003b2ab1-config-file\") pod \"limitador-limitador-54d6d988d5-672sq\" (UID: \"1dc80bb7-afa4-438c-86a6-4360003b2ab1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-672sq" Jul 08 19:14:31.133217 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:31.133200 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzbwn\" (UniqueName: \"kubernetes.io/projected/1dc80bb7-afa4-438c-86a6-4360003b2ab1-kube-api-access-fzbwn\") pod \"limitador-limitador-54d6d988d5-672sq\" (UID: \"1dc80bb7-afa4-438c-86a6-4360003b2ab1\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-672sq" Jul 08 19:14:31.216008 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:31.215963 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-nm986"] Jul 08 19:14:31.218962 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:31.218939 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-nm986" Jul 08 19:14:31.221252 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:31.221232 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bkhd4\"" Jul 08 19:14:31.229725 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:31.229704 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-nm986"] Jul 08 19:14:31.238295 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:31.238272 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-672sq" Jul 08 19:14:31.326244 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:31.326192 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcvzb\" (UniqueName: \"kubernetes.io/projected/0233de7d-aff3-48a0-9513-61f16176b9a5-kube-api-access-qcvzb\") pod \"authorino-58cd955675-nm986\" (UID: \"0233de7d-aff3-48a0-9513-61f16176b9a5\") " pod="kuadrant-system/authorino-58cd955675-nm986" Jul 08 19:14:31.376353 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:31.376326 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:14:31.378578 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:14:31.378548 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dc80bb7_afa4_438c_86a6_4360003b2ab1.slice/crio-a97763e6b2c90becd6dec54ce77ddbbfb0291a1edf5084628c2eb1dc505a6706 WatchSource:0}: Error finding container a97763e6b2c90becd6dec54ce77ddbbfb0291a1edf5084628c2eb1dc505a6706: Status 404 returned error can't find the container with id a97763e6b2c90becd6dec54ce77ddbbfb0291a1edf5084628c2eb1dc505a6706 Jul 08 19:14:31.427260 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:31.427225 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcvzb\" (UniqueName: \"kubernetes.io/projected/0233de7d-aff3-48a0-9513-61f16176b9a5-kube-api-access-qcvzb\") pod \"authorino-58cd955675-nm986\" (UID: \"0233de7d-aff3-48a0-9513-61f16176b9a5\") " pod="kuadrant-system/authorino-58cd955675-nm986" Jul 08 19:14:31.436221 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:31.436187 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcvzb\" (UniqueName: \"kubernetes.io/projected/0233de7d-aff3-48a0-9513-61f16176b9a5-kube-api-access-qcvzb\") pod \"authorino-58cd955675-nm986\" (UID: \"0233de7d-aff3-48a0-9513-61f16176b9a5\") " pod="kuadrant-system/authorino-58cd955675-nm986" Jul 08 19:14:31.528144 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:31.528113 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-nm986" Jul 08 19:14:31.671153 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:31.671120 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-nm986"] Jul 08 19:14:31.672163 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:14:31.672140 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0233de7d_aff3_48a0_9513_61f16176b9a5.slice/crio-78fb0eb4ab2ae793a164a933bfa731fc05898afa5d0253161a51005e83d6e844 WatchSource:0}: Error finding container 78fb0eb4ab2ae793a164a933bfa731fc05898afa5d0253161a51005e83d6e844: Status 404 returned error can't find the container with id 78fb0eb4ab2ae793a164a933bfa731fc05898afa5d0253161a51005e83d6e844 Jul 08 19:14:31.962867 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:31.962725 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-nm986" event={"ID":"0233de7d-aff3-48a0-9513-61f16176b9a5","Type":"ContainerStarted","Data":"78fb0eb4ab2ae793a164a933bfa731fc05898afa5d0253161a51005e83d6e844"} Jul 08 19:14:31.964688 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:31.964658 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-672sq" event={"ID":"1dc80bb7-afa4-438c-86a6-4360003b2ab1","Type":"ContainerStarted","Data":"a97763e6b2c90becd6dec54ce77ddbbfb0291a1edf5084628c2eb1dc505a6706"} Jul 08 19:14:35.994610 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:35.994557 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-672sq" event={"ID":"1dc80bb7-afa4-438c-86a6-4360003b2ab1","Type":"ContainerStarted","Data":"d9fbc75807f584c1ed346f4fbaec70bb05c913cbf4417942a1913e0db6b438cd"} Jul 08 19:14:35.995052 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:35.994643 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-672sq" Jul 08 19:14:35.996051 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:35.996030 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-nm986" event={"ID":"0233de7d-aff3-48a0-9513-61f16176b9a5","Type":"ContainerStarted","Data":"4b1d27eac9fd4cf1874a185f690a4f14435a18c932c64c53c84b22c2a3ace58f"} Jul 08 19:14:36.014623 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:36.014514 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-672sq" podStartSLOduration=1.646798865 podStartE2EDuration="6.01450104s" totalImagesPullingTime="4.367702175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:14:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:14:36.013247847 +0000 UTC m=+550.191381865" watchObservedRunningTime="2026-07-08 19:14:36.01450104 +0000 UTC m=+550.192635045" Jul 08 19:14:36.031840 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:36.031795 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-nm986" podStartSLOduration=0.899109065 podStartE2EDuration="5.031781748s" totalImagesPullingTime="4.132672683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:14:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:14:36.028817855 +0000 UTC m=+550.206951863" watchObservedRunningTime="2026-07-08 19:14:36.031781748 +0000 UTC m=+550.209915754" Jul 08 19:14:36.195799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:36.195759 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-nm986"] Jul 08 19:14:38.005948 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:38.005911 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-nm986" podUID="0233de7d-aff3-48a0-9513-61f16176b9a5" containerName="authorino" containerID="cri-o://4b1d27eac9fd4cf1874a185f690a4f14435a18c932c64c53c84b22c2a3ace58f" gracePeriod=30 Jul 08 19:14:38.243563 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:38.243507 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-nm986" Jul 08 19:14:38.397345 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:38.397323 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qcvzb\" (UniqueName: \"kubernetes.io/projected/0233de7d-aff3-48a0-9513-61f16176b9a5-kube-api-access-qcvzb\") pod \"0233de7d-aff3-48a0-9513-61f16176b9a5\" (UID: \"0233de7d-aff3-48a0-9513-61f16176b9a5\") " Jul 08 19:14:38.399577 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:38.399520 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0233de7d-aff3-48a0-9513-61f16176b9a5-kube-api-access-qcvzb" (OuterVolumeSpecName: "kube-api-access-qcvzb") pod "0233de7d-aff3-48a0-9513-61f16176b9a5" (UID: "0233de7d-aff3-48a0-9513-61f16176b9a5"). InnerVolumeSpecName "kube-api-access-qcvzb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:14:38.498250 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:38.498212 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qcvzb\" (UniqueName: \"kubernetes.io/projected/0233de7d-aff3-48a0-9513-61f16176b9a5-kube-api-access-qcvzb\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:14:39.012442 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:39.012407 2560 generic.go:356] "Generic (PLEG): container finished" podID="0233de7d-aff3-48a0-9513-61f16176b9a5" containerID="4b1d27eac9fd4cf1874a185f690a4f14435a18c932c64c53c84b22c2a3ace58f" exitCode=0 Jul 08 19:14:39.012932 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:39.012468 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-nm986" Jul 08 19:14:39.012932 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:39.012499 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-nm986" event={"ID":"0233de7d-aff3-48a0-9513-61f16176b9a5","Type":"ContainerDied","Data":"4b1d27eac9fd4cf1874a185f690a4f14435a18c932c64c53c84b22c2a3ace58f"} Jul 08 19:14:39.012932 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:39.012562 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-nm986" event={"ID":"0233de7d-aff3-48a0-9513-61f16176b9a5","Type":"ContainerDied","Data":"78fb0eb4ab2ae793a164a933bfa731fc05898afa5d0253161a51005e83d6e844"} Jul 08 19:14:39.012932 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:39.012581 2560 scope.go:117] "RemoveContainer" containerID="4b1d27eac9fd4cf1874a185f690a4f14435a18c932c64c53c84b22c2a3ace58f" Jul 08 19:14:39.021816 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:39.021801 2560 scope.go:117] "RemoveContainer" containerID="4b1d27eac9fd4cf1874a185f690a4f14435a18c932c64c53c84b22c2a3ace58f" Jul 08 19:14:39.022085 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:14:39.022066 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b1d27eac9fd4cf1874a185f690a4f14435a18c932c64c53c84b22c2a3ace58f\": container with ID starting with 4b1d27eac9fd4cf1874a185f690a4f14435a18c932c64c53c84b22c2a3ace58f not found: ID does not exist" containerID="4b1d27eac9fd4cf1874a185f690a4f14435a18c932c64c53c84b22c2a3ace58f" Jul 08 19:14:39.022132 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:39.022095 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b1d27eac9fd4cf1874a185f690a4f14435a18c932c64c53c84b22c2a3ace58f"} err="failed to get container status \"4b1d27eac9fd4cf1874a185f690a4f14435a18c932c64c53c84b22c2a3ace58f\": rpc error: code = NotFound desc = could not find container \"4b1d27eac9fd4cf1874a185f690a4f14435a18c932c64c53c84b22c2a3ace58f\": container with ID starting with 4b1d27eac9fd4cf1874a185f690a4f14435a18c932c64c53c84b22c2a3ace58f not found: ID does not exist" Jul 08 19:14:39.033877 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:39.033850 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-nm986"] Jul 08 19:14:39.038290 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:39.038268 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-nm986"] Jul 08 19:14:40.354450 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:40.354417 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0233de7d-aff3-48a0-9513-61f16176b9a5" path="/var/lib/kubelet/pods/0233de7d-aff3-48a0-9513-61f16176b9a5/volumes" Jul 08 19:14:47.002334 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:14:47.002301 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-672sq" Jul 08 19:15:04.045445 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:04.045410 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-64wcc"] Jul 08 19:15:04.045989 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:04.045800 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0233de7d-aff3-48a0-9513-61f16176b9a5" containerName="authorino" Jul 08 19:15:04.045989 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:04.045812 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0233de7d-aff3-48a0-9513-61f16176b9a5" containerName="authorino" Jul 08 19:15:04.045989 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:04.045866 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0233de7d-aff3-48a0-9513-61f16176b9a5" containerName="authorino" Jul 08 19:15:04.047968 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:04.047954 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-64wcc" Jul 08 19:15:04.050729 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:04.050700 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bkhd4\"" Jul 08 19:15:04.056857 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:04.056835 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-64wcc"] Jul 08 19:15:04.124659 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:04.124624 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk5zg\" (UniqueName: \"kubernetes.io/projected/8c800324-50d1-4a2d-844e-959c97c8525d-kube-api-access-xk5zg\") pod \"authorino-5d4b67c446-64wcc\" (UID: \"8c800324-50d1-4a2d-844e-959c97c8525d\") " pod="kuadrant-system/authorino-5d4b67c446-64wcc" Jul 08 19:15:04.225854 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:04.225820 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk5zg\" (UniqueName: \"kubernetes.io/projected/8c800324-50d1-4a2d-844e-959c97c8525d-kube-api-access-xk5zg\") pod \"authorino-5d4b67c446-64wcc\" (UID: \"8c800324-50d1-4a2d-844e-959c97c8525d\") " pod="kuadrant-system/authorino-5d4b67c446-64wcc" Jul 08 19:15:04.240436 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:04.240413 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk5zg\" (UniqueName: \"kubernetes.io/projected/8c800324-50d1-4a2d-844e-959c97c8525d-kube-api-access-xk5zg\") pod \"authorino-5d4b67c446-64wcc\" (UID: \"8c800324-50d1-4a2d-844e-959c97c8525d\") " pod="kuadrant-system/authorino-5d4b67c446-64wcc" Jul 08 19:15:04.323311 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:04.323211 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-64wcc"] Jul 08 19:15:04.323561 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:04.323512 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-64wcc" Jul 08 19:15:04.473606 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:04.473578 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-64wcc"] Jul 08 19:15:04.475359 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:15:04.475332 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8c800324_50d1_4a2d_844e_959c97c8525d.slice/crio-ae3a1cb040b197febea428a89b4b71f91d61d64a60f79de7708b429aa220519b WatchSource:0}: Error finding container ae3a1cb040b197febea428a89b4b71f91d61d64a60f79de7708b429aa220519b: Status 404 returned error can't find the container with id ae3a1cb040b197febea428a89b4b71f91d61d64a60f79de7708b429aa220519b Jul 08 19:15:05.147207 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:05.147170 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-64wcc" event={"ID":"8c800324-50d1-4a2d-844e-959c97c8525d","Type":"ContainerStarted","Data":"ae3a1cb040b197febea428a89b4b71f91d61d64a60f79de7708b429aa220519b"} Jul 08 19:15:06.153897 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:06.153856 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-64wcc" event={"ID":"8c800324-50d1-4a2d-844e-959c97c8525d","Type":"ContainerStarted","Data":"3ddb51a5fe1ec0afdfa0ef27ad97023f3d75fe02da22f5da0e0dedbf8482a5d9"} Jul 08 19:15:06.154339 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:06.153917 2560 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-64wcc" podUID="8c800324-50d1-4a2d-844e-959c97c8525d" containerName="authorino" containerID="cri-o://3ddb51a5fe1ec0afdfa0ef27ad97023f3d75fe02da22f5da0e0dedbf8482a5d9" gracePeriod=30 Jul 08 19:15:06.172074 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:06.172027 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-64wcc" podStartSLOduration=1.16555265 podStartE2EDuration="2.172011895s" totalImagesPullingTime="1.006459245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:15:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:15:06.17064403 +0000 UTC m=+580.348778036" watchObservedRunningTime="2026-07-08 19:15:06.172011895 +0000 UTC m=+580.350145901" Jul 08 19:15:06.445520 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:06.445488 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-64wcc" Jul 08 19:15:06.546990 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:06.546963 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xk5zg\" (UniqueName: \"kubernetes.io/projected/8c800324-50d1-4a2d-844e-959c97c8525d-kube-api-access-xk5zg\") pod \"8c800324-50d1-4a2d-844e-959c97c8525d\" (UID: \"8c800324-50d1-4a2d-844e-959c97c8525d\") " Jul 08 19:15:06.549118 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:06.549085 2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c800324-50d1-4a2d-844e-959c97c8525d-kube-api-access-xk5zg" (OuterVolumeSpecName: "kube-api-access-xk5zg") pod "8c800324-50d1-4a2d-844e-959c97c8525d" (UID: "8c800324-50d1-4a2d-844e-959c97c8525d"). InnerVolumeSpecName "kube-api-access-xk5zg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 19:15:06.648263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:06.648228 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xk5zg\" (UniqueName: \"kubernetes.io/projected/8c800324-50d1-4a2d-844e-959c97c8525d-kube-api-access-xk5zg\") on node \"ip-10-0-128-38.ec2.internal\" DevicePath \"\"" Jul 08 19:15:07.160691 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:07.160652 2560 generic.go:356] "Generic (PLEG): container finished" podID="8c800324-50d1-4a2d-844e-959c97c8525d" containerID="3ddb51a5fe1ec0afdfa0ef27ad97023f3d75fe02da22f5da0e0dedbf8482a5d9" exitCode=0 Jul 08 19:15:07.161129 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:07.160698 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-64wcc" Jul 08 19:15:07.161129 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:07.160742 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-64wcc" event={"ID":"8c800324-50d1-4a2d-844e-959c97c8525d","Type":"ContainerDied","Data":"3ddb51a5fe1ec0afdfa0ef27ad97023f3d75fe02da22f5da0e0dedbf8482a5d9"} Jul 08 19:15:07.161129 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:07.160778 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-64wcc" event={"ID":"8c800324-50d1-4a2d-844e-959c97c8525d","Type":"ContainerDied","Data":"ae3a1cb040b197febea428a89b4b71f91d61d64a60f79de7708b429aa220519b"} Jul 08 19:15:07.161129 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:07.160801 2560 scope.go:117] "RemoveContainer" containerID="3ddb51a5fe1ec0afdfa0ef27ad97023f3d75fe02da22f5da0e0dedbf8482a5d9" Jul 08 19:15:07.170038 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:07.170019 2560 scope.go:117] "RemoveContainer" containerID="3ddb51a5fe1ec0afdfa0ef27ad97023f3d75fe02da22f5da0e0dedbf8482a5d9" Jul 08 19:15:07.170326 ip-10-0-128-38 kubenswrapper[2560]: E0708 19:15:07.170307 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ddb51a5fe1ec0afdfa0ef27ad97023f3d75fe02da22f5da0e0dedbf8482a5d9\": container with ID starting with 3ddb51a5fe1ec0afdfa0ef27ad97023f3d75fe02da22f5da0e0dedbf8482a5d9 not found: ID does not exist" containerID="3ddb51a5fe1ec0afdfa0ef27ad97023f3d75fe02da22f5da0e0dedbf8482a5d9" Jul 08 19:15:07.170385 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:07.170340 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ddb51a5fe1ec0afdfa0ef27ad97023f3d75fe02da22f5da0e0dedbf8482a5d9"} err="failed to get container status \"3ddb51a5fe1ec0afdfa0ef27ad97023f3d75fe02da22f5da0e0dedbf8482a5d9\": rpc error: code = NotFound desc = could not find container \"3ddb51a5fe1ec0afdfa0ef27ad97023f3d75fe02da22f5da0e0dedbf8482a5d9\": container with ID starting with 3ddb51a5fe1ec0afdfa0ef27ad97023f3d75fe02da22f5da0e0dedbf8482a5d9 not found: ID does not exist" Jul 08 19:15:07.189412 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:07.189381 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-64wcc"] Jul 08 19:15:07.193172 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:07.193150 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-64wcc"] Jul 08 19:15:08.354698 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:08.354665 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c800324-50d1-4a2d-844e-959c97c8525d" path="/var/lib/kubelet/pods/8c800324-50d1-4a2d-844e-959c97c8525d/volumes" Jul 08 19:15:45.008003 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:15:45.007966 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:16:07.850782 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:16:07.850742 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:16:27.648260 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:16:27.648225 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:16:34.948317 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:16:34.948278 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:16:58.102926 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:16:58.102893 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:17:18.855135 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:17:18.855063 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:17:29.150820 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:17:29.150783 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:18:50.391886 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:18:50.391803 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-576b8cf4f-6t668"] Jul 08 19:18:50.392352 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:18:50.392198 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c800324-50d1-4a2d-844e-959c97c8525d" containerName="authorino" Jul 08 19:18:50.392352 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:18:50.392210 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c800324-50d1-4a2d-844e-959c97c8525d" containerName="authorino" Jul 08 19:18:50.392352 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:18:50.392269 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c800324-50d1-4a2d-844e-959c97c8525d" containerName="authorino" Jul 08 19:18:50.394198 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:18:50.394180 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-576b8cf4f-6t668" Jul 08 19:18:50.397502 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:18:50.397478 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 08 19:18:50.397684 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:18:50.397478 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bkhd4\"" Jul 08 19:18:50.402920 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:18:50.402891 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-576b8cf4f-6t668"] Jul 08 19:18:50.427272 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:18:50.427246 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dd9f273d-d8d6-4bb1-a439-2f4637b3f4c0-tls-cert\") pod \"authorino-576b8cf4f-6t668\" (UID: \"dd9f273d-d8d6-4bb1-a439-2f4637b3f4c0\") " pod="kuadrant-system/authorino-576b8cf4f-6t668" Jul 08 19:18:50.427433 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:18:50.427290 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4qwm\" (UniqueName: \"kubernetes.io/projected/dd9f273d-d8d6-4bb1-a439-2f4637b3f4c0-kube-api-access-f4qwm\") pod \"authorino-576b8cf4f-6t668\" (UID: \"dd9f273d-d8d6-4bb1-a439-2f4637b3f4c0\") " pod="kuadrant-system/authorino-576b8cf4f-6t668" Jul 08 19:18:50.527805 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:18:50.527768 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4qwm\" (UniqueName: \"kubernetes.io/projected/dd9f273d-d8d6-4bb1-a439-2f4637b3f4c0-kube-api-access-f4qwm\") pod \"authorino-576b8cf4f-6t668\" (UID: \"dd9f273d-d8d6-4bb1-a439-2f4637b3f4c0\") " pod="kuadrant-system/authorino-576b8cf4f-6t668" Jul 08 19:18:50.528021 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:18:50.527862 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dd9f273d-d8d6-4bb1-a439-2f4637b3f4c0-tls-cert\") pod \"authorino-576b8cf4f-6t668\" (UID: \"dd9f273d-d8d6-4bb1-a439-2f4637b3f4c0\") " pod="kuadrant-system/authorino-576b8cf4f-6t668" Jul 08 19:18:50.530519 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:18:50.530496 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/dd9f273d-d8d6-4bb1-a439-2f4637b3f4c0-tls-cert\") pod \"authorino-576b8cf4f-6t668\" (UID: \"dd9f273d-d8d6-4bb1-a439-2f4637b3f4c0\") " pod="kuadrant-system/authorino-576b8cf4f-6t668" Jul 08 19:18:50.536294 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:18:50.536270 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4qwm\" (UniqueName: \"kubernetes.io/projected/dd9f273d-d8d6-4bb1-a439-2f4637b3f4c0-kube-api-access-f4qwm\") pod \"authorino-576b8cf4f-6t668\" (UID: \"dd9f273d-d8d6-4bb1-a439-2f4637b3f4c0\") " pod="kuadrant-system/authorino-576b8cf4f-6t668" Jul 08 19:18:50.704229 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:18:50.704138 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-576b8cf4f-6t668" Jul 08 19:18:50.837799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:18:50.837769 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-576b8cf4f-6t668"] Jul 08 19:18:50.839227 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:18:50.839199 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd9f273d_d8d6_4bb1_a439_2f4637b3f4c0.slice/crio-943157c30153c228fda779d1cc179c787386e089e5396411259570938ba73515 WatchSource:0}: Error finding container 943157c30153c228fda779d1cc179c787386e089e5396411259570938ba73515: Status 404 returned error can't find the container with id 943157c30153c228fda779d1cc179c787386e089e5396411259570938ba73515 Jul 08 19:18:50.840506 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:18:50.840487 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:18:51.315542 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:18:51.315482 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-576b8cf4f-6t668" event={"ID":"dd9f273d-d8d6-4bb1-a439-2f4637b3f4c0","Type":"ContainerStarted","Data":"943157c30153c228fda779d1cc179c787386e089e5396411259570938ba73515"} Jul 08 19:18:52.322609 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:18:52.322563 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-576b8cf4f-6t668" event={"ID":"dd9f273d-d8d6-4bb1-a439-2f4637b3f4c0","Type":"ContainerStarted","Data":"5fc34b97bb1db0ac05a597d6b7602d4525dd468caf7396ff5b54872a125e830b"} Jul 08 19:18:52.358762 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:18:52.358712 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-576b8cf4f-6t668" podStartSLOduration=1.809772964 podStartE2EDuration="2.35869189s" totalImagesPullingTime="548.918926ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:18:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:18:52.35805742 +0000 UTC m=+806.536191484" watchObservedRunningTime="2026-07-08 19:18:52.35869189 +0000 UTC m=+806.536825896" Jul 08 19:19:22.739374 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:19:22.739337 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:19:31.948844 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:19:31.948807 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:19:43.837451 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:19:43.837410 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:19:52.552725 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:19:52.552693 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:20:03.548965 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:20:03.548924 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:20:30.915805 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:20:30.915771 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:20:42.740053 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:20:42.740014 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:21:18.555356 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:21:18.555315 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:21:28.241728 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:21:28.241683 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:21:38.758773 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:21:38.758731 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:21:46.646656 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:21:46.646560 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:21:58.460483 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:21:58.460444 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:22:10.645509 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:22:10.645467 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:22:21.439660 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:22:21.439626 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:22:31.535316 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:22:31.535274 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:22:53.341650 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:22:53.341602 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:23:05.736519 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:23:05.736484 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:23:28.756858 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:23:28.756770 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:23:39.644029 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:23:39.643991 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:23:55.544991 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:23:55.544948 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:24:05.143117 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:24:05.143080 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:24:14.945700 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:24:14.945660 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:24:24.841284 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:24:24.841246 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:24:33.902344 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:24:33.902303 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:26:23.969042 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:26:23.968999 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:26:33.232519 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:26:33.232478 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:26:46.650581 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:26:46.650542 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:26:59.153832 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:26:59.153798 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:27:11.042054 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:27:11.042020 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:27:20.761357 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:27:20.756887 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:27:29.836263 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:27:29.836223 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:27:41.140284 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:27:41.140243 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:27:50.845257 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:27:50.845166 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:28:01.642669 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:28:01.642631 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:28:13.344512 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:28:13.344469 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:28:35.661506 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:28:35.661467 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:28:44.345267 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:28:44.345232 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:28:55.841112 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:28:55.841069 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:29:05.451921 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:29:05.451880 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:29:22.242021 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:29:22.241946 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:29:34.136044 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:29:34.136005 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:29:43.939389 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:29:43.939353 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:29:51.341883 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:29:51.341847 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:30:02.942265 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:30:02.942230 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:30:12.935844 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:30:12.935805 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:30:27.129772 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:30:27.129733 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:30:41.745170 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:30:41.745126 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:30:51.440566 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:30:51.440446 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:30:58.537891 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:30:58.537856 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:31:08.127653 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:31:08.127617 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:31:15.634513 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:31:15.634472 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:31:32.240392 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:31:32.240350 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:31:41.843644 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:31:41.843607 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:31:51.062510 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:31:51.062474 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:31:57.842625 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:31:57.842586 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:32:11.334085 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:32:11.334048 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:32:20.149476 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:32:20.149388 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:32:30.446251 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:32:30.446213 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:32:50.845463 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:32:50.845421 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:33:00.739841 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:33:00.739806 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:34:39.337556 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:34:39.337492 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:34:48.648161 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:34:48.648119 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:34:57.745298 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:34:57.745260 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:35:10.251330 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:35:10.251277 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:35:42.246176 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:35:42.246129 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:35:57.248704 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:35:57.248666 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:36:11.840460 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:36:11.840410 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:36:25.042647 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:36:25.042613 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:37:10.852399 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:37:10.852355 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:37:20.800826 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:37:20.800791 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:37:34.653670 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:37:34.653633 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:38:22.845761 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:38:22.845687 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:38:39.048933 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:38:39.048891 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:38:53.053125 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:38:53.053084 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:39:34.043430 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:39:34.043387 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:39:58.747052 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:39:58.747006 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:40:39.336393 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:40:39.336353 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:40:54.240667 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:40:54.240628 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:41:11.741710 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:41:11.741673 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:41:25.948406 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:41:25.948321 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:41:34.646688 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:41:34.646649 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:41:45.144656 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:41:45.144619 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:44:46.336640 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:44:46.336543 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:44:56.692085 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:44:56.692056 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-576b8cf4f-6t668_dd9f273d-d8d6-4bb1-a439-2f4637b3f4c0/authorino/0.log" Jul 08 19:44:57.937767 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:44:57.937736 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-672sq"] Jul 08 19:45:01.645878 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:01.645849 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-p2b8c_9987fb5f-d765-4f19-b236-039ec3cd50aa/server/0.log" Jul 08 19:45:01.762764 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:01.762727 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pnwjc_65100a2c-1816-4008-a01b-b4761767a40d/manager/0.log" Jul 08 19:45:01.879060 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:01.879028 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6667fbd97f-b2kdw_442cb34b-f9b9-4ed7-87a4-0199a56cb2a9/copy-manifests/0.log" Jul 08 19:45:01.886326 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:01.886297 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6667fbd97f-b2kdw_442cb34b-f9b9-4ed7-87a4-0199a56cb2a9/manager/0.log" Jul 08 19:45:02.912142 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:02.912108 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m_9a154f14-66b0-47d2-a6a9-1339729905ab/util/0.log" Jul 08 19:45:02.918376 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:02.918345 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m_9a154f14-66b0-47d2-a6a9-1339729905ab/pull/0.log" Jul 08 19:45:02.924513 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:02.924488 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m_9a154f14-66b0-47d2-a6a9-1339729905ab/extract/0.log" Jul 08 19:45:03.037073 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:03.037047 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x_39fab212-2aa5-419f-acbb-7a96d731cac6/util/0.log" Jul 08 19:45:03.044819 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:03.044786 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x_39fab212-2aa5-419f-acbb-7a96d731cac6/pull/0.log" Jul 08 19:45:03.051386 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:03.051363 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x_39fab212-2aa5-419f-acbb-7a96d731cac6/extract/0.log" Jul 08 19:45:03.165251 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:03.165177 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr_d213b05c-939a-46b5-bcf5-38baab0e3e7b/util/0.log" Jul 08 19:45:03.171418 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:03.171395 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr_d213b05c-939a-46b5-bcf5-38baab0e3e7b/pull/0.log" Jul 08 19:45:03.176756 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:03.176733 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr_d213b05c-939a-46b5-bcf5-38baab0e3e7b/extract/0.log" Jul 08 19:45:03.296046 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:03.296018 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd_d95cfc77-c5bb-4427-bbd6-d07de14710ae/util/0.log" Jul 08 19:45:03.303113 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:03.303079 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd_d95cfc77-c5bb-4427-bbd6-d07de14710ae/pull/0.log" Jul 08 19:45:03.310084 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:03.310051 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd_d95cfc77-c5bb-4427-bbd6-d07de14710ae/extract/0.log" Jul 08 19:45:03.441563 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:03.441458 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-576b8cf4f-6t668_dd9f273d-d8d6-4bb1-a439-2f4637b3f4c0/authorino/0.log" Jul 08 19:45:03.790704 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:03.790625 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-5tp4x_226ade13-1862-4b6f-a4f7-e24ea6fb44ad/kuadrant-console-plugin/0.log" Jul 08 19:45:04.155081 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:04.155060 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-672sq_1dc80bb7-afa4-438c-86a6-4360003b2ab1/limitador/0.log" Jul 08 19:45:04.646277 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:04.646250 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4bd4zz_b415f298-0354-4f5a-9204-0dfcced0e3c7/istio-proxy/0.log" Jul 08 19:45:04.996923 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:04.996842 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-565878cbdc-lqb2c_06db37a8-78bb-47af-8d73-815c55f9f88a/kube-auth-proxy/0.log" Jul 08 19:45:11.328928 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:11.328894 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sjlc8/must-gather-x5ghm"] Jul 08 19:45:11.332799 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:11.332775 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sjlc8/must-gather-x5ghm" Jul 08 19:45:11.335286 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:11.335259 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sjlc8\"/\"default-dockercfg-mwzxk\"" Jul 08 19:45:11.336051 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:11.336032 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sjlc8\"/\"openshift-service-ca.crt\"" Jul 08 19:45:11.336161 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:11.336035 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sjlc8\"/\"kube-root-ca.crt\"" Jul 08 19:45:11.341702 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:11.341679 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sjlc8/must-gather-x5ghm"] Jul 08 19:45:11.470521 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:11.470481 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/aa6eb0c4-2d83-41db-9447-58f735ae830d-must-gather-output\") pod \"must-gather-x5ghm\" (UID: \"aa6eb0c4-2d83-41db-9447-58f735ae830d\") " pod="openshift-must-gather-sjlc8/must-gather-x5ghm" Jul 08 19:45:11.470704 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:11.470618 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gknvb\" (UniqueName: \"kubernetes.io/projected/aa6eb0c4-2d83-41db-9447-58f735ae830d-kube-api-access-gknvb\") pod \"must-gather-x5ghm\" (UID: \"aa6eb0c4-2d83-41db-9447-58f735ae830d\") " pod="openshift-must-gather-sjlc8/must-gather-x5ghm" Jul 08 19:45:11.571437 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:11.571404 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/aa6eb0c4-2d83-41db-9447-58f735ae830d-must-gather-output\") pod \"must-gather-x5ghm\" (UID: \"aa6eb0c4-2d83-41db-9447-58f735ae830d\") " pod="openshift-must-gather-sjlc8/must-gather-x5ghm" Jul 08 19:45:11.571631 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:11.571480 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gknvb\" (UniqueName: \"kubernetes.io/projected/aa6eb0c4-2d83-41db-9447-58f735ae830d-kube-api-access-gknvb\") pod \"must-gather-x5ghm\" (UID: \"aa6eb0c4-2d83-41db-9447-58f735ae830d\") " pod="openshift-must-gather-sjlc8/must-gather-x5ghm" Jul 08 19:45:11.571771 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:11.571752 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/aa6eb0c4-2d83-41db-9447-58f735ae830d-must-gather-output\") pod \"must-gather-x5ghm\" (UID: \"aa6eb0c4-2d83-41db-9447-58f735ae830d\") " pod="openshift-must-gather-sjlc8/must-gather-x5ghm" Jul 08 19:45:11.579719 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:11.579657 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gknvb\" (UniqueName: \"kubernetes.io/projected/aa6eb0c4-2d83-41db-9447-58f735ae830d-kube-api-access-gknvb\") pod \"must-gather-x5ghm\" (UID: \"aa6eb0c4-2d83-41db-9447-58f735ae830d\") " pod="openshift-must-gather-sjlc8/must-gather-x5ghm" Jul 08 19:45:11.643303 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:11.643264 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sjlc8/must-gather-x5ghm" Jul 08 19:45:11.781639 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:11.781609 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sjlc8/must-gather-x5ghm"] Jul 08 19:45:11.783290 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:45:11.783258 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa6eb0c4_2d83_41db_9447_58f735ae830d.slice/crio-1e3d782e1967e728372d1357d9d1a3d48e281a0901a5e5f0845e8186bd8fda36 WatchSource:0}: Error finding container 1e3d782e1967e728372d1357d9d1a3d48e281a0901a5e5f0845e8186bd8fda36: Status 404 returned error can't find the container with id 1e3d782e1967e728372d1357d9d1a3d48e281a0901a5e5f0845e8186bd8fda36 Jul 08 19:45:11.785314 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:11.785289 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 19:45:12.703919 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:12.703879 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sjlc8/must-gather-x5ghm" event={"ID":"aa6eb0c4-2d83-41db-9447-58f735ae830d","Type":"ContainerStarted","Data":"1e3d782e1967e728372d1357d9d1a3d48e281a0901a5e5f0845e8186bd8fda36"} Jul 08 19:45:13.713165 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:13.713131 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sjlc8/must-gather-x5ghm" event={"ID":"aa6eb0c4-2d83-41db-9447-58f735ae830d","Type":"ContainerStarted","Data":"d7c341588a57b4667db212f9350f5f7eed0bffdba944a06c32aad837b7a209d6"} Jul 08 19:45:13.713669 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:13.713175 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sjlc8/must-gather-x5ghm" event={"ID":"aa6eb0c4-2d83-41db-9447-58f735ae830d","Type":"ContainerStarted","Data":"8c479b2a7b60d530ae9272528d8b9572087f89e6b2786a1a0d329cb92512eb35"} Jul 08 19:45:13.732013 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:13.731952 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sjlc8/must-gather-x5ghm" podStartSLOduration=1.693640599 podStartE2EDuration="2.731927694s" totalImagesPullingTime="1.038287095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 19:45:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:45:13.731510581 +0000 UTC m=+2387.909644589" watchObservedRunningTime="2026-07-08 19:45:13.731927694 +0000 UTC m=+2387.910061702" Jul 08 19:45:14.549693 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:14.549657 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fmbfk_77822693-2416-4400-87cf-4619ac479e79/global-pull-secret-syncer/0.log" Jul 08 19:45:14.720051 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:14.720019 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wndp8_69978d76-a5ad-4ebf-9d22-8335ec387429/konnectivity-agent/0.log" Jul 08 19:45:14.746305 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:14.746272 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-38.ec2.internal_301ed78f06a8f545f44b18fb2b42ca58/haproxy/0.log" Jul 08 19:45:19.103922 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:19.103872 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m_9a154f14-66b0-47d2-a6a9-1339729905ab/extract/0.log" Jul 08 19:45:19.127234 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:19.127187 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m_9a154f14-66b0-47d2-a6a9-1339729905ab/util/0.log" Jul 08 19:45:19.156088 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:19.156049 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759npn6m_9a154f14-66b0-47d2-a6a9-1339729905ab/pull/0.log" Jul 08 19:45:19.186086 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:19.186050 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x_39fab212-2aa5-419f-acbb-7a96d731cac6/extract/0.log" Jul 08 19:45:19.209903 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:19.209856 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x_39fab212-2aa5-419f-acbb-7a96d731cac6/util/0.log" Jul 08 19:45:19.236645 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:19.236611 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0cp92x_39fab212-2aa5-419f-acbb-7a96d731cac6/pull/0.log" Jul 08 19:45:19.276438 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:19.276407 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr_d213b05c-939a-46b5-bcf5-38baab0e3e7b/extract/0.log" Jul 08 19:45:19.301001 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:19.300968 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr_d213b05c-939a-46b5-bcf5-38baab0e3e7b/util/0.log" Jul 08 19:45:19.329768 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:19.329736 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mpqcr_d213b05c-939a-46b5-bcf5-38baab0e3e7b/pull/0.log" Jul 08 19:45:19.369285 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:19.369185 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd_d95cfc77-c5bb-4427-bbd6-d07de14710ae/extract/0.log" Jul 08 19:45:19.423391 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:19.423356 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd_d95cfc77-c5bb-4427-bbd6-d07de14710ae/util/0.log" Jul 08 19:45:19.475928 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:19.475893 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1dfpfd_d95cfc77-c5bb-4427-bbd6-d07de14710ae/pull/0.log" Jul 08 19:45:19.943681 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:19.943632 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-576b8cf4f-6t668_dd9f273d-d8d6-4bb1-a439-2f4637b3f4c0/authorino/0.log" Jul 08 19:45:20.034891 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:20.034667 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-5tp4x_226ade13-1862-4b6f-a4f7-e24ea6fb44ad/kuadrant-console-plugin/0.log" Jul 08 19:45:20.212410 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:20.212303 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-672sq_1dc80bb7-afa4-438c-86a6-4360003b2ab1/limitador/0.log" Jul 08 19:45:21.932092 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:21.932058 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7e23d33c-4709-4320-8359-dea6305b5869/alertmanager/0.log" Jul 08 19:45:21.959361 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:21.959328 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7e23d33c-4709-4320-8359-dea6305b5869/config-reloader/0.log" Jul 08 19:45:21.989094 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:21.989065 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7e23d33c-4709-4320-8359-dea6305b5869/kube-rbac-proxy-web/0.log" Jul 08 19:45:22.012939 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:22.012909 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7e23d33c-4709-4320-8359-dea6305b5869/kube-rbac-proxy/0.log" Jul 08 19:45:22.034848 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:22.034794 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7e23d33c-4709-4320-8359-dea6305b5869/kube-rbac-proxy-metric/0.log" Jul 08 19:45:22.057186 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:22.057163 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7e23d33c-4709-4320-8359-dea6305b5869/prom-label-proxy/0.log" Jul 08 19:45:22.077808 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:22.077729 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7e23d33c-4709-4320-8359-dea6305b5869/init-config-reloader/0.log" Jul 08 19:45:22.221883 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:22.221861 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7df89d47dc-rkktq_4ac7ccb3-0375-4b54-befa-a72aea0de69b/metrics-server/0.log" Jul 08 19:45:22.276933 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:22.276906 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r9427_c1272e21-4e8e-476f-b367-093abb99cc1d/node-exporter/0.log" Jul 08 19:45:22.300909 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:22.300883 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r9427_c1272e21-4e8e-476f-b367-093abb99cc1d/kube-rbac-proxy/0.log" Jul 08 19:45:22.321760 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:22.321735 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r9427_c1272e21-4e8e-476f-b367-093abb99cc1d/init-textfile/0.log" Jul 08 19:45:22.740112 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:22.740077 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-w5gd7_34ed99b8-94b0-43a1-9895-a171af9a4e6f/prometheus-operator/0.log" Jul 08 19:45:22.757059 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:22.757026 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-w5gd7_34ed99b8-94b0-43a1-9895-a171af9a4e6f/kube-rbac-proxy/0.log" Jul 08 19:45:22.784898 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:22.784869 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-jkccg_2ee4c41f-9323-452f-b3f4-90ddfeec954e/prometheus-operator-admission-webhook/0.log" Jul 08 19:45:22.819373 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:22.819339 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-68c889fb65-ns8jm_927d1f81-c5c6-4639-a1fa-0be3e6d58c07/telemeter-client/0.log" Jul 08 19:45:22.848414 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:22.848381 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-68c889fb65-ns8jm_927d1f81-c5c6-4639-a1fa-0be3e6d58c07/reload/0.log" Jul 08 19:45:22.870489 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:22.870446 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-68c889fb65-ns8jm_927d1f81-c5c6-4639-a1fa-0be3e6d58c07/kube-rbac-proxy/0.log" Jul 08 19:45:22.918582 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:22.918554 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bbbd84965-f65nq_58f67a80-c6f3-4e56-a472-71946c9be733/thanos-query/0.log" Jul 08 19:45:22.938477 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:22.938445 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bbbd84965-f65nq_58f67a80-c6f3-4e56-a472-71946c9be733/kube-rbac-proxy-web/0.log" Jul 08 19:45:22.958826 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:22.958795 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bbbd84965-f65nq_58f67a80-c6f3-4e56-a472-71946c9be733/kube-rbac-proxy/0.log" Jul 08 19:45:22.980366 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:22.980335 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bbbd84965-f65nq_58f67a80-c6f3-4e56-a472-71946c9be733/prom-label-proxy/0.log" Jul 08 19:45:23.001458 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:23.001382 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bbbd84965-f65nq_58f67a80-c6f3-4e56-a472-71946c9be733/kube-rbac-proxy-rules/0.log" Jul 08 19:45:23.022164 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:23.022135 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6bbbd84965-f65nq_58f67a80-c6f3-4e56-a472-71946c9be733/kube-rbac-proxy-metrics/0.log" Jul 08 19:45:24.543092 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:24.542940 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh"] Jul 08 19:45:24.553607 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:24.553569 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" Jul 08 19:45:24.565683 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:24.565636 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh"] Jul 08 19:45:24.608555 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:24.607590 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9qzt\" (UniqueName: \"kubernetes.io/projected/63f77d6f-4a6e-4299-aeb0-91a901417659-kube-api-access-q9qzt\") pod \"perf-node-gather-daemonset-7wqzh\" (UID: \"63f77d6f-4a6e-4299-aeb0-91a901417659\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" Jul 08 19:45:24.608555 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:24.607654 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63f77d6f-4a6e-4299-aeb0-91a901417659-lib-modules\") pod \"perf-node-gather-daemonset-7wqzh\" (UID: \"63f77d6f-4a6e-4299-aeb0-91a901417659\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" Jul 08 19:45:24.608555 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:24.607717 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/63f77d6f-4a6e-4299-aeb0-91a901417659-podres\") pod \"perf-node-gather-daemonset-7wqzh\" (UID: \"63f77d6f-4a6e-4299-aeb0-91a901417659\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" Jul 08 19:45:24.608555 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:24.607770 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/63f77d6f-4a6e-4299-aeb0-91a901417659-proc\") pod \"perf-node-gather-daemonset-7wqzh\" (UID: \"63f77d6f-4a6e-4299-aeb0-91a901417659\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" Jul 08 19:45:24.608555 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:24.607827 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63f77d6f-4a6e-4299-aeb0-91a901417659-sys\") pod \"perf-node-gather-daemonset-7wqzh\" (UID: \"63f77d6f-4a6e-4299-aeb0-91a901417659\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" Jul 08 19:45:24.709009 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:24.708964 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/63f77d6f-4a6e-4299-aeb0-91a901417659-podres\") pod \"perf-node-gather-daemonset-7wqzh\" (UID: \"63f77d6f-4a6e-4299-aeb0-91a901417659\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" Jul 08 19:45:24.709211 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:24.709049 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/63f77d6f-4a6e-4299-aeb0-91a901417659-proc\") pod \"perf-node-gather-daemonset-7wqzh\" (UID: \"63f77d6f-4a6e-4299-aeb0-91a901417659\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" Jul 08 19:45:24.709211 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:24.709106 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63f77d6f-4a6e-4299-aeb0-91a901417659-sys\") pod \"perf-node-gather-daemonset-7wqzh\" (UID: \"63f77d6f-4a6e-4299-aeb0-91a901417659\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" Jul 08 19:45:24.709336 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:24.709211 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/63f77d6f-4a6e-4299-aeb0-91a901417659-sys\") pod \"perf-node-gather-daemonset-7wqzh\" (UID: \"63f77d6f-4a6e-4299-aeb0-91a901417659\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" Jul 08 19:45:24.709336 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:24.709217 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/63f77d6f-4a6e-4299-aeb0-91a901417659-proc\") pod \"perf-node-gather-daemonset-7wqzh\" (UID: \"63f77d6f-4a6e-4299-aeb0-91a901417659\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" Jul 08 19:45:24.709336 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:24.709274 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9qzt\" (UniqueName: \"kubernetes.io/projected/63f77d6f-4a6e-4299-aeb0-91a901417659-kube-api-access-q9qzt\") pod \"perf-node-gather-daemonset-7wqzh\" (UID: \"63f77d6f-4a6e-4299-aeb0-91a901417659\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" Jul 08 19:45:24.709336 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:24.709327 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63f77d6f-4a6e-4299-aeb0-91a901417659-lib-modules\") pod \"perf-node-gather-daemonset-7wqzh\" (UID: \"63f77d6f-4a6e-4299-aeb0-91a901417659\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" Jul 08 19:45:24.709667 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:24.709512 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/63f77d6f-4a6e-4299-aeb0-91a901417659-lib-modules\") pod \"perf-node-gather-daemonset-7wqzh\" (UID: \"63f77d6f-4a6e-4299-aeb0-91a901417659\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" Jul 08 19:45:24.709667 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:24.709600 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/63f77d6f-4a6e-4299-aeb0-91a901417659-podres\") pod \"perf-node-gather-daemonset-7wqzh\" (UID: \"63f77d6f-4a6e-4299-aeb0-91a901417659\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" Jul 08 19:45:24.719924 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:24.719852 2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9qzt\" (UniqueName: \"kubernetes.io/projected/63f77d6f-4a6e-4299-aeb0-91a901417659-kube-api-access-q9qzt\") pod \"perf-node-gather-daemonset-7wqzh\" (UID: \"63f77d6f-4a6e-4299-aeb0-91a901417659\") " pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" Jul 08 19:45:24.872780 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:24.872694 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" Jul 08 19:45:25.059080 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:25.059052 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh"] Jul 08 19:45:25.060970 ip-10-0-128-38 kubenswrapper[2560]: W0708 19:45:25.060936 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod63f77d6f_4a6e_4299_aeb0_91a901417659.slice/crio-e2ecfebad52582d306fe239eb8aba7617af234de8e6ffc1e61ca9cb2c1cf60af WatchSource:0}: Error finding container e2ecfebad52582d306fe239eb8aba7617af234de8e6ffc1e61ca9cb2c1cf60af: Status 404 returned error can't find the container with id e2ecfebad52582d306fe239eb8aba7617af234de8e6ffc1e61ca9cb2c1cf60af Jul 08 19:45:25.424112 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:25.424045 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-gjhnf_ab32487f-34c8-4010-a1d7-aea7c4ab6378/download-server/0.log" Jul 08 19:45:25.859868 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:25.859774 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" event={"ID":"63f77d6f-4a6e-4299-aeb0-91a901417659","Type":"ContainerStarted","Data":"6174c0e5cb3214f284dc511d2cd959e4f05274704d2fa428abb3d4f9f2381a7a"} Jul 08 19:45:25.860396 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:25.860371 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" event={"ID":"63f77d6f-4a6e-4299-aeb0-91a901417659","Type":"ContainerStarted","Data":"e2ecfebad52582d306fe239eb8aba7617af234de8e6ffc1e61ca9cb2c1cf60af"} Jul 08 19:45:25.860571 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:25.860520 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" Jul 08 19:45:25.878060 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:25.877998 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" podStartSLOduration=1.87797645 podStartE2EDuration="1.87797645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 19:45:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 19:45:25.875765764 +0000 UTC m=+2400.053899782" watchObservedRunningTime="2026-07-08 19:45:25.87797645 +0000 UTC m=+2400.056110457" Jul 08 19:45:27.122634 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:27.122598 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bjxfb_6e9b6b99-5daa-4d32-81fd-9b3aa823171b/dns/0.log" Jul 08 19:45:27.154447 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:27.154413 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bjxfb_6e9b6b99-5daa-4d32-81fd-9b3aa823171b/kube-rbac-proxy/0.log" Jul 08 19:45:27.303432 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:27.303408 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-sg8ph_a7d66e59-7d94-4a00-86f4-fbb8dfe65280/dns-node-resolver/0.log" Jul 08 19:45:27.887019 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:27.886988 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xs94k_04ab4794-25e2-4c82-9e44-b460a3993f26/node-ca/0.log" Jul 08 19:45:28.811404 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:28.811369 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4bd4zz_b415f298-0354-4f5a-9204-0dfcced0e3c7/istio-proxy/0.log" Jul 08 19:45:28.954227 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:28.954199 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-565878cbdc-lqb2c_06db37a8-78bb-47af-8d73-815c55f9f88a/kube-auth-proxy/0.log" Jul 08 19:45:29.842137 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:29.842108 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b42qc_987f6244-9238-41f0-9ba9-55b5815c995e/serve-healthcheck-canary/0.log" Jul 08 19:45:30.453717 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:30.453682 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xs97g_098a2451-3786-4094-9a48-19d8b3804a9f/kube-rbac-proxy/0.log" Jul 08 19:45:30.473848 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:30.473820 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xs97g_098a2451-3786-4094-9a48-19d8b3804a9f/exporter/0.log" Jul 08 19:45:30.496644 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:30.496609 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xs97g_098a2451-3786-4094-9a48-19d8b3804a9f/extractor/0.log" Jul 08 19:45:32.887357 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:32.887327 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sjlc8/perf-node-gather-daemonset-7wqzh" Jul 08 19:45:32.889388 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:32.889366 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-p2b8c_9987fb5f-d765-4f19-b236-039ec3cd50aa/server/0.log" Jul 08 19:45:32.989878 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:32.989825 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pnwjc_65100a2c-1816-4008-a01b-b4761767a40d/manager/0.log" Jul 08 19:45:33.014162 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:33.014134 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6667fbd97f-b2kdw_442cb34b-f9b9-4ed7-87a4-0199a56cb2a9/manager/0.log" Jul 08 19:45:33.034977 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:33.034936 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6667fbd97f-b2kdw_442cb34b-f9b9-4ed7-87a4-0199a56cb2a9/copy-manifests/0.log" Jul 08 19:45:34.500695 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:34.500665 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6f5b6f7879-lrlh5_07293172-4f4a-4ebd-9fc1-89503717a147/manager/0.log" Jul 08 19:45:34.563361 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:34.563332 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-426bn_f62a0f2d-a379-4601-892d-6c9fb10cd3fe/openshift-lws-operator/0.log" Jul 08 19:45:35.071743 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:35.071658 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4bd4zz_b415f298-0354-4f5a-9204-0dfcced0e3c7/istio-proxy/0.log" Jul 08 19:45:35.198851 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:35.198817 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-565878cbdc-lqb2c_06db37a8-78bb-47af-8d73-815c55f9f88a/kube-auth-proxy/0.log" Jul 08 19:45:36.499843 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:36.499810 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4bd4zz_b415f298-0354-4f5a-9204-0dfcced0e3c7/istio-proxy/0.log" Jul 08 19:45:36.616070 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:36.616038 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-565878cbdc-lqb2c_06db37a8-78bb-47af-8d73-815c55f9f88a/kube-auth-proxy/0.log" Jul 08 19:45:39.774451 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:39.774422 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-fdtk5_9154ff70-49b0-4fbf-bdfe-cdc6d0bbad80/migrator/0.log" Jul 08 19:45:39.792695 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:39.792670 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-fdtk5_9154ff70-49b0-4fbf-bdfe-cdc6d0bbad80/graceful-termination/0.log" Jul 08 19:45:41.171109 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:41.171074 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5sxcv_c80f1ff4-3d7b-49dc-9320-e6ecba209dc1/kube-multus/0.log" Jul 08 19:45:41.224474 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:41.224425 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5wxhc_6349a4f6-297b-4718-9981-b1dbc8415f79/kube-multus-additional-cni-plugins/0.log" Jul 08 19:45:41.244607 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:41.244575 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5wxhc_6349a4f6-297b-4718-9981-b1dbc8415f79/egress-router-binary-copy/0.log" Jul 08 19:45:41.266332 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:41.266301 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5wxhc_6349a4f6-297b-4718-9981-b1dbc8415f79/cni-plugins/0.log" Jul 08 19:45:41.287457 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:41.287430 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5wxhc_6349a4f6-297b-4718-9981-b1dbc8415f79/bond-cni-plugin/0.log" Jul 08 19:45:41.307415 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:41.307389 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5wxhc_6349a4f6-297b-4718-9981-b1dbc8415f79/routeoverride-cni/0.log" Jul 08 19:45:41.326383 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:41.326354 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5wxhc_6349a4f6-297b-4718-9981-b1dbc8415f79/whereabouts-cni-bincopy/0.log" Jul 08 19:45:41.346028 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:41.346004 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5wxhc_6349a4f6-297b-4718-9981-b1dbc8415f79/whereabouts-cni/0.log" Jul 08 19:45:41.703006 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:41.702977 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2bqn6_3981d8c6-c84c-439a-bf4b-d9834812c8bb/network-metrics-daemon/0.log" Jul 08 19:45:41.720611 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:41.720582 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2bqn6_3981d8c6-c84c-439a-bf4b-d9834812c8bb/kube-rbac-proxy/0.log" Jul 08 19:45:42.812340 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:42.812311 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nklcn_426767c6-42b0-4249-b95c-d0487aa20faa/ovn-controller/0.log" Jul 08 19:45:42.827133 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:42.827109 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nklcn_426767c6-42b0-4249-b95c-d0487aa20faa/ovn-acl-logging/0.log" Jul 08 19:45:42.848199 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:42.848170 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nklcn_426767c6-42b0-4249-b95c-d0487aa20faa/kube-rbac-proxy-node/0.log" Jul 08 19:45:42.866778 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:42.866754 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nklcn_426767c6-42b0-4249-b95c-d0487aa20faa/kube-rbac-proxy-ovn-metrics/0.log" Jul 08 19:45:42.884897 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:42.884872 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nklcn_426767c6-42b0-4249-b95c-d0487aa20faa/northd/0.log" Jul 08 19:45:42.902431 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:42.902409 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nklcn_426767c6-42b0-4249-b95c-d0487aa20faa/nbdb/0.log" Jul 08 19:45:42.921842 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:42.921777 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nklcn_426767c6-42b0-4249-b95c-d0487aa20faa/sbdb/0.log" Jul 08 19:45:42.995736 ip-10-0-128-38 kubenswrapper[2560]: I0708 19:45:42.995699 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nklcn_426767c6-42b0-4249-b95c-d0487aa20faa/ovnkube-controller/0.log"