Jul 06 16:38:05.719764 ip-10-0-135-40 systemd[1]: Starting Kubernetes Kubelet... Jul 06 16:38:06.071953 ip-10-0-135-40 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 16:38:06.071953 ip-10-0-135-40 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 16:38:06.071953 ip-10-0-135-40 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 16:38:06.071953 ip-10-0-135-40 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 16:38:06.071953 ip-10-0-135-40 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 16:38:06.073555 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.073402 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 16:38:06.077151 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077130 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 16:38:06.077151 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077146 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 16:38:06.077151 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077151 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 16:38:06.077151 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077155 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 16:38:06.077151 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077158 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 16:38:06.077403 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077163 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 16:38:06.077403 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077167 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 16:38:06.077403 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077170 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 16:38:06.077403 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077174 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 16:38:06.077403 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077179 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 16:38:06.077403 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077182 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 16:38:06.077403 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077187 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 16:38:06.077403 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077191 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 16:38:06.077403 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077197 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 16:38:06.077403 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077202 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 16:38:06.077403 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077206 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 16:38:06.077403 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077210 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 16:38:06.077403 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077215 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 16:38:06.077403 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077219 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 16:38:06.077403 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077223 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 16:38:06.077403 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077227 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 16:38:06.077403 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077239 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 16:38:06.077403 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077243 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 16:38:06.077403 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077247 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 16:38:06.078245 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077251 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 16:38:06.078245 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077255 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 16:38:06.078245 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077259 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 16:38:06.078245 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077263 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 16:38:06.078245 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077267 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 16:38:06.078245 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077271 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 16:38:06.078245 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077276 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 16:38:06.078245 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077280 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 16:38:06.078245 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077285 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 16:38:06.078245 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077297 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 16:38:06.078245 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077302 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 16:38:06.078245 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077306 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 16:38:06.078245 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077310 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 16:38:06.078245 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077314 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 16:38:06.078245 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077320 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 16:38:06.078245 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077325 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 16:38:06.078245 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077329 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 16:38:06.078245 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077333 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 16:38:06.078245 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077338 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 16:38:06.078245 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077342 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 16:38:06.079089 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077346 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 16:38:06.079089 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077350 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 16:38:06.079089 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077354 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 16:38:06.079089 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077358 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 16:38:06.079089 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077363 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 16:38:06.079089 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077367 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 16:38:06.079089 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077371 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 16:38:06.079089 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077375 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 16:38:06.079089 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077379 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 16:38:06.079089 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077384 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 16:38:06.079089 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077389 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 16:38:06.079089 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077393 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 16:38:06.079089 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077397 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 16:38:06.079089 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077401 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 16:38:06.079089 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077406 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 06 16:38:06.079089 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077410 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 16:38:06.079089 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077413 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 16:38:06.079089 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077422 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 16:38:06.079089 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077427 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 16:38:06.079089 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077432 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 16:38:06.079922 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077437 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 16:38:06.079922 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077442 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 16:38:06.079922 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077447 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 16:38:06.079922 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077451 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 16:38:06.079922 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077456 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 16:38:06.079922 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077460 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 16:38:06.079922 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077483 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 16:38:06.079922 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077488 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 16:38:06.079922 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077492 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 16:38:06.079922 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077496 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 16:38:06.079922 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077501 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 16:38:06.079922 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077505 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 16:38:06.079922 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077509 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 16:38:06.079922 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077513 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 16:38:06.079922 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077517 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 16:38:06.079922 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077521 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 16:38:06.079922 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077525 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 16:38:06.079922 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077530 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 16:38:06.079922 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077534 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 16:38:06.079922 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077538 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 16:38:06.080761 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077542 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 16:38:06.080761 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077546 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 16:38:06.080761 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077550 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 16:38:06.080761 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077555 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 16:38:06.080761 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077560 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 16:38:06.080761 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077564 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 16:38:06.080761 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077571 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 16:38:06.080761 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077577 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 16:38:06.080761 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077582 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 16:38:06.080761 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077586 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 16:38:06.080761 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077590 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 16:38:06.080761 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077594 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 16:38:06.080761 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077598 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 16:38:06.080761 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077602 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 16:38:06.080761 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.077606 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 16:38:06.080761 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078746 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 16:38:06.080761 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078758 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 16:38:06.080761 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078762 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 06 16:38:06.080761 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078768 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 16:38:06.080761 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078773 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 16:38:06.081625 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078778 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 16:38:06.081625 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078782 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 16:38:06.081625 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078787 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 16:38:06.081625 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078791 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 16:38:06.081625 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078795 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 16:38:06.081625 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078800 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 16:38:06.081625 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078804 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 16:38:06.081625 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078808 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 16:38:06.081625 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078812 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 16:38:06.081625 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078816 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 16:38:06.081625 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078820 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 16:38:06.081625 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078825 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 16:38:06.081625 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078829 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 16:38:06.081625 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078836 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 16:38:06.081625 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078842 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 16:38:06.081625 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078847 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 16:38:06.081625 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078852 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 16:38:06.081625 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078859 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 16:38:06.081625 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078863 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 16:38:06.081625 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078868 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 16:38:06.082216 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078872 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 16:38:06.082216 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078876 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 16:38:06.082216 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078880 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 16:38:06.082216 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078886 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 16:38:06.082216 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078891 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 16:38:06.082216 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078895 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 16:38:06.082216 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078900 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 16:38:06.082216 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078904 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 16:38:06.082216 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078908 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 16:38:06.082216 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078912 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 16:38:06.082216 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078917 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 16:38:06.082216 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078922 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 16:38:06.082216 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078926 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 16:38:06.082216 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078930 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 16:38:06.082216 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078934 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 16:38:06.082216 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078938 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 16:38:06.082216 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078942 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 16:38:06.082216 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078946 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 16:38:06.082216 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078951 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 16:38:06.082216 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078955 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 16:38:06.082793 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078959 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 16:38:06.082793 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078963 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 16:38:06.082793 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078967 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 16:38:06.082793 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078971 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 16:38:06.082793 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078976 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 16:38:06.082793 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078980 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 16:38:06.082793 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078984 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 16:38:06.082793 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078988 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 16:38:06.082793 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078992 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 16:38:06.082793 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.078997 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 16:38:06.082793 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079001 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 16:38:06.082793 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079005 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 16:38:06.082793 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079009 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 16:38:06.082793 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079013 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 16:38:06.082793 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079017 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 16:38:06.082793 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079021 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 16:38:06.082793 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079025 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 16:38:06.082793 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079030 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 16:38:06.082793 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079034 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 16:38:06.083368 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079038 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 16:38:06.083368 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079042 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 16:38:06.083368 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079047 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 16:38:06.083368 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079051 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 16:38:06.083368 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079055 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 16:38:06.083368 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079060 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 16:38:06.083368 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079066 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 16:38:06.083368 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079071 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 16:38:06.083368 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079076 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 16:38:06.083368 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079081 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 16:38:06.083368 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079087 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 16:38:06.083368 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079091 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 16:38:06.083368 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079096 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 16:38:06.083368 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079100 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 16:38:06.083368 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079104 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 16:38:06.083368 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079107 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 16:38:06.083368 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079112 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 16:38:06.083368 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079116 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 16:38:06.083368 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079121 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 16:38:06.083968 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079125 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 16:38:06.083968 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079129 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 16:38:06.083968 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079134 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 16:38:06.083968 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079139 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 16:38:06.083968 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079144 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 16:38:06.083968 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079148 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 16:38:06.083968 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079152 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 16:38:06.083968 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079156 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 16:38:06.083968 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079160 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 16:38:06.083968 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079164 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 16:38:06.083968 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079168 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 16:38:06.083968 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079172 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 16:38:06.083968 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079177 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 16:38:06.083968 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079181 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 16:38:06.083968 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079185 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 16:38:06.083968 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079192 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 16:38:06.083968 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079781 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 16:38:06.083968 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079789 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 16:38:06.083968 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079793 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 16:38:06.083968 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079798 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 16:38:06.084443 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079803 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 16:38:06.084443 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079808 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 16:38:06.084443 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079812 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 16:38:06.084443 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079817 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 16:38:06.084443 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079821 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 16:38:06.084443 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079825 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 16:38:06.084443 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079830 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 16:38:06.084443 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079834 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 16:38:06.084443 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079838 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 16:38:06.084443 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079842 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 16:38:06.084443 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079846 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 16:38:06.084443 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079851 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 16:38:06.084443 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079855 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 16:38:06.084443 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079859 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 16:38:06.084443 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079863 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 16:38:06.084443 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079867 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 16:38:06.084443 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079872 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 16:38:06.084443 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079876 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 16:38:06.084443 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079880 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 16:38:06.084996 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079887 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 16:38:06.084996 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079893 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 16:38:06.084996 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079898 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 16:38:06.084996 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079903 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 16:38:06.084996 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079907 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 16:38:06.084996 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079911 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 16:38:06.084996 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079915 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 16:38:06.084996 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079919 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 16:38:06.084996 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079926 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 16:38:06.084996 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079930 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 16:38:06.084996 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079936 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 16:38:06.084996 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079942 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 06 16:38:06.084996 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079946 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 16:38:06.084996 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079951 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 16:38:06.084996 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079956 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 16:38:06.084996 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079961 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 16:38:06.084996 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079965 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 16:38:06.084996 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079971 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 16:38:06.084996 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079975 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 16:38:06.085459 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079979 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 16:38:06.085459 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079983 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 16:38:06.085459 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079987 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 16:38:06.085459 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079991 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 16:38:06.085459 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.079995 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 16:38:06.085459 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080000 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 16:38:06.085459 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080004 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 16:38:06.085459 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080008 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 16:38:06.085459 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080013 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 16:38:06.085459 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080017 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 16:38:06.085459 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080021 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 16:38:06.085459 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080026 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 16:38:06.085459 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080030 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 16:38:06.085459 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080034 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 16:38:06.085459 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080038 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 16:38:06.085459 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080042 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 16:38:06.085459 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080047 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 16:38:06.085459 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080051 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 16:38:06.085459 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080055 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 16:38:06.085459 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080059 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 16:38:06.086130 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080063 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 16:38:06.086130 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080069 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 16:38:06.086130 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080073 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 16:38:06.086130 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080077 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 16:38:06.086130 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080081 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 16:38:06.086130 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080086 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 16:38:06.086130 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080090 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 16:38:06.086130 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080094 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 16:38:06.086130 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080099 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 16:38:06.086130 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080103 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 16:38:06.086130 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080107 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 16:38:06.086130 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080111 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 16:38:06.086130 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080115 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 16:38:06.086130 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080119 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 16:38:06.086130 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080124 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 16:38:06.086130 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080128 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 16:38:06.086130 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080132 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 16:38:06.086130 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080136 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 16:38:06.086130 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080140 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 16:38:06.086130 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080146 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 16:38:06.086935 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080149 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 16:38:06.086935 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080153 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 16:38:06.086935 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080157 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 16:38:06.086935 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080162 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 16:38:06.086935 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080167 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 16:38:06.086935 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080171 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 16:38:06.086935 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080175 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 16:38:06.086935 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080179 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 16:38:06.086935 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080183 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 16:38:06.086935 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080188 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 16:38:06.086935 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080193 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 16:38:06.086935 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080197 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 16:38:06.086935 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080201 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 16:38:06.086935 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080207 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 16:38:06.086935 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080211 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 16:38:06.086935 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080215 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 16:38:06.086935 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.080219 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 16:38:06.086935 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080314 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 16:38:06.086935 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080324 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 16:38:06.086935 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080333 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 16:38:06.086935 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080340 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080346 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080351 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080357 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080363 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080368 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080373 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080378 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080383 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080388 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080393 2580 flags.go:64] FLAG: --cgroup-root="" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080397 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080402 2580 flags.go:64] FLAG: --client-ca-file="" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080406 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080411 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080417 2580 flags.go:64] FLAG: --cluster-domain="" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080421 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080428 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080433 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080438 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080444 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080449 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080454 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 16:38:06.087805 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080459 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080464 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080486 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080491 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080497 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080501 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080507 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080512 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080517 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080521 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080526 2580 flags.go:64] FLAG: --enable-server="true" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080531 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080537 2580 flags.go:64] FLAG: --event-burst="100" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080542 2580 flags.go:64] FLAG: --event-qps="50" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080546 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080551 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080556 2580 flags.go:64] FLAG: --eviction-hard="" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080568 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080573 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080578 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080582 2580 flags.go:64] FLAG: --eviction-soft="" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080587 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080591 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080596 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080603 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 16:38:06.088595 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080607 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080612 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080618 2580 flags.go:64] FLAG: --feature-gates="" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080625 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080630 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080635 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080640 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080644 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080649 2580 flags.go:64] FLAG: --help="false" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080657 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-135-40.ec2.internal" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080662 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080667 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080671 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080677 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080682 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080687 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080692 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080696 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080701 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080705 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080711 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080715 2580 flags.go:64] FLAG: --kube-reserved="" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080719 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080723 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 16:38:06.089368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080729 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080733 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080738 2580 flags.go:64] FLAG: --lock-file="" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080742 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080747 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080752 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080760 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080766 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080771 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080775 2580 flags.go:64] FLAG: --logging-format="text" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080779 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080786 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080790 2580 flags.go:64] FLAG: --manifest-url="" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080795 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080801 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080806 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080813 2580 flags.go:64] FLAG: --max-pods="110" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080818 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080823 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080827 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080831 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080837 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080842 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080846 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080856 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 16:38:06.090139 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080861 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080866 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080871 2580 flags.go:64] FLAG: --pod-cidr="" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080875 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080883 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080888 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080893 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080897 2580 flags.go:64] FLAG: --port="10250" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080902 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080906 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05d8c9ae2ff4755dd" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080911 2580 flags.go:64] FLAG: --qos-reserved="" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080916 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080921 2580 flags.go:64] FLAG: --register-node="true" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080926 2580 flags.go:64] FLAG: --register-with-taints="" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080931 2580 flags.go:64] FLAG: --registry-burst="10" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080938 2580 flags.go:64] FLAG: --registry-qps="5" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080942 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080947 2580 flags.go:64] FLAG: --reserved-memory="" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080952 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080958 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080963 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080968 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080972 2580 flags.go:64] FLAG: --runonce="false" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080977 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080982 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 16:38:06.090767 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080987 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080991 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.080996 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081001 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081006 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081011 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081016 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081021 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081025 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081030 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081035 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081040 2580 flags.go:64] FLAG: --system-cgroups="" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081044 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081053 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081057 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081062 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081068 2580 flags.go:64] FLAG: --tls-min-version="" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081073 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081078 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081083 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081087 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081092 2580 flags.go:64] FLAG: --v="2" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081100 2580 flags.go:64] FLAG: --version="false" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081106 2580 flags.go:64] FLAG: --vmodule="" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081111 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 16:38:06.091365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.081117 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 16:38:06.091979 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081291 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 16:38:06.091979 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081299 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 16:38:06.091979 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081305 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 16:38:06.091979 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081309 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 16:38:06.091979 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081314 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 16:38:06.091979 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081319 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 16:38:06.091979 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081324 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 16:38:06.091979 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081328 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 16:38:06.091979 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081332 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 16:38:06.091979 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081338 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 16:38:06.091979 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081343 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 16:38:06.091979 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081348 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 16:38:06.091979 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081352 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 16:38:06.091979 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081357 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 16:38:06.091979 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081361 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 16:38:06.091979 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081365 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 16:38:06.091979 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081369 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 16:38:06.091979 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081373 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 16:38:06.091979 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081378 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 16:38:06.092440 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081382 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 16:38:06.092440 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081387 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 16:38:06.092440 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081391 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 06 16:38:06.092440 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081395 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 16:38:06.092440 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081401 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 16:38:06.092440 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081407 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 16:38:06.092440 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081413 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 16:38:06.092440 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081417 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 16:38:06.092440 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081422 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 16:38:06.092440 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081429 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 16:38:06.092440 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081433 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 16:38:06.092440 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081437 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 16:38:06.092440 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081441 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 16:38:06.092440 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081446 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 16:38:06.092440 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081450 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 16:38:06.092440 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081455 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 16:38:06.092440 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081459 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 16:38:06.092440 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081463 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 16:38:06.092440 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081485 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 16:38:06.092954 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081491 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 16:38:06.092954 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081494 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 16:38:06.092954 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081499 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 16:38:06.092954 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081503 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 16:38:06.092954 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081507 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 16:38:06.092954 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081511 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 16:38:06.092954 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081516 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 16:38:06.092954 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081520 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 16:38:06.092954 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081524 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 16:38:06.092954 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081528 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 16:38:06.092954 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081532 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 16:38:06.092954 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081537 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 16:38:06.092954 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081541 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 16:38:06.092954 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081546 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 16:38:06.092954 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081550 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 16:38:06.092954 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081554 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 16:38:06.092954 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081559 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 16:38:06.092954 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081563 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 16:38:06.092954 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081568 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 16:38:06.092954 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081572 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 16:38:06.093428 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081576 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 16:38:06.093428 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081580 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 16:38:06.093428 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081588 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 16:38:06.093428 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081594 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 16:38:06.093428 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081599 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 16:38:06.093428 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081604 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 16:38:06.093428 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081609 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 16:38:06.093428 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081613 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 16:38:06.093428 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081618 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 16:38:06.093428 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081623 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 16:38:06.093428 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081627 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 16:38:06.093428 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081633 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 16:38:06.093428 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081637 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 16:38:06.093428 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081641 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 16:38:06.093428 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081645 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 16:38:06.093428 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081649 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 16:38:06.093428 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081654 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 16:38:06.093428 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081658 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 16:38:06.093428 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081663 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 16:38:06.093911 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081667 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 16:38:06.093911 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081671 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 16:38:06.093911 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081675 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 16:38:06.093911 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081680 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 16:38:06.093911 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081684 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 16:38:06.093911 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081688 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 16:38:06.093911 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081692 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 16:38:06.093911 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081696 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 16:38:06.093911 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081701 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 16:38:06.093911 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081705 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 16:38:06.093911 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081709 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 16:38:06.093911 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081714 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 16:38:06.093911 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081718 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 16:38:06.093911 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081722 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 16:38:06.093911 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081726 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 16:38:06.093911 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081732 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 16:38:06.093911 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081736 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 16:38:06.093911 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081740 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 16:38:06.093911 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081745 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 16:38:06.093911 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081750 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 16:38:06.094393 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081754 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 16:38:06.094393 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.081758 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 16:38:06.094393 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.082422 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 16:38:06.094393 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.091115 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 16:38:06.094393 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.091135 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 16:38:06.094393 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.092633 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 16:38:06.094393 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.092640 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 16:38:06.094393 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092695 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 16:38:06.094393 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092701 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 16:38:06.094393 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092704 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 16:38:06.094393 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092706 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 16:38:06.094393 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092709 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 16:38:06.094393 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092713 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 16:38:06.094393 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092717 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 16:38:06.094834 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092720 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 16:38:06.094834 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092722 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 16:38:06.094834 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092725 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 16:38:06.094834 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092728 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 16:38:06.094834 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092731 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 16:38:06.094834 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092734 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 16:38:06.094834 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092736 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 16:38:06.094834 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092739 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 16:38:06.094834 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092742 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 16:38:06.094834 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092744 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 16:38:06.094834 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092747 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 16:38:06.094834 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092750 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 16:38:06.094834 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092752 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 16:38:06.094834 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092755 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 16:38:06.094834 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092758 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 16:38:06.094834 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092760 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 16:38:06.094834 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092765 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 16:38:06.094834 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092769 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 16:38:06.094834 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092772 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 16:38:06.095301 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092775 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 16:38:06.095301 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092778 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 16:38:06.095301 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092781 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 16:38:06.095301 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092785 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 16:38:06.095301 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092788 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 16:38:06.095301 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092790 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 16:38:06.095301 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092793 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 16:38:06.095301 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092795 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 16:38:06.095301 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092798 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 16:38:06.095301 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092800 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 16:38:06.095301 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092803 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 16:38:06.095301 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092805 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 16:38:06.095301 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092808 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 16:38:06.095301 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092810 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 16:38:06.095301 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092813 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 16:38:06.095301 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092815 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 16:38:06.095301 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092818 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 16:38:06.095301 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092820 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 16:38:06.095301 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092823 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 16:38:06.095772 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092826 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 16:38:06.095772 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092829 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 16:38:06.095772 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092832 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 16:38:06.095772 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092834 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 16:38:06.095772 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092837 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 16:38:06.095772 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092840 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 16:38:06.095772 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092842 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 16:38:06.095772 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092845 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 16:38:06.095772 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092847 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 16:38:06.095772 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092850 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 16:38:06.095772 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092852 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 16:38:06.095772 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092855 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 16:38:06.095772 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092857 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 16:38:06.095772 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092860 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 16:38:06.095772 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092862 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 16:38:06.095772 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092865 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 16:38:06.095772 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092868 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 16:38:06.095772 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092871 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 16:38:06.095772 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092873 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 16:38:06.095772 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092876 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 16:38:06.096251 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092878 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 16:38:06.096251 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092881 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 16:38:06.096251 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092883 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 16:38:06.096251 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092886 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 16:38:06.096251 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092888 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 16:38:06.096251 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092891 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 16:38:06.096251 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092893 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 16:38:06.096251 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092896 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 16:38:06.096251 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092898 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 16:38:06.096251 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092901 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 16:38:06.096251 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092903 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 16:38:06.096251 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092906 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 16:38:06.096251 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092908 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 16:38:06.096251 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092911 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 16:38:06.096251 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092914 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 16:38:06.096251 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092917 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 16:38:06.096251 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092920 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 16:38:06.096251 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092924 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 16:38:06.096251 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092927 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 16:38:06.096720 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092930 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 16:38:06.096720 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092932 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 16:38:06.096720 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092935 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 16:38:06.096720 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092938 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 16:38:06.096720 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092941 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 16:38:06.096720 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092943 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 06 16:38:06.096720 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092946 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 16:38:06.096720 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092948 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 16:38:06.096720 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092950 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 16:38:06.096720 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092953 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 16:38:06.096720 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092956 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 16:38:06.096720 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092959 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 16:38:06.096720 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092961 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 16:38:06.096720 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092964 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 16:38:06.096720 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.092967 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 16:38:06.097085 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.092971 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 16:38:06.097085 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093074 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 16:38:06.097085 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093079 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 16:38:06.097085 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093082 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 16:38:06.097085 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093085 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 16:38:06.097085 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093088 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 16:38:06.097085 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093090 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 16:38:06.097085 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093093 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 16:38:06.097085 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093096 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 16:38:06.097085 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093099 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 16:38:06.097085 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093102 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 16:38:06.097085 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093107 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 16:38:06.097085 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093109 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 16:38:06.097085 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093113 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 16:38:06.097547 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093116 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 16:38:06.097547 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093118 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 16:38:06.097547 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093121 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 16:38:06.097547 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093123 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 16:38:06.097547 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093126 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 16:38:06.097547 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093128 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 16:38:06.097547 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093130 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 16:38:06.097547 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093133 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 16:38:06.097547 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093135 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 16:38:06.097547 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093138 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 16:38:06.097547 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093140 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 16:38:06.097547 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093143 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 16:38:06.097547 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093145 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 16:38:06.097547 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093148 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 16:38:06.097547 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093151 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 16:38:06.097547 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093153 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 16:38:06.097547 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093156 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 16:38:06.097547 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093158 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 16:38:06.097547 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093161 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 16:38:06.097547 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093163 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 16:38:06.098022 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093166 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 16:38:06.098022 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093168 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 16:38:06.098022 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093171 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 16:38:06.098022 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093173 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 16:38:06.098022 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093176 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 16:38:06.098022 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093178 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 16:38:06.098022 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093181 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 16:38:06.098022 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093183 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 16:38:06.098022 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093186 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 16:38:06.098022 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093188 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 16:38:06.098022 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093191 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 16:38:06.098022 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093194 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 16:38:06.098022 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093198 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 16:38:06.098022 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093201 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 16:38:06.098022 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093203 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 16:38:06.098022 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093206 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 16:38:06.098022 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093209 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 16:38:06.098022 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093211 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 16:38:06.098022 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093214 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 16:38:06.098511 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093217 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 16:38:06.098511 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093220 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 16:38:06.098511 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093222 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 16:38:06.098511 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093225 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 16:38:06.098511 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093228 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 16:38:06.098511 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093230 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 16:38:06.098511 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093233 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 16:38:06.098511 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093236 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 16:38:06.098511 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093238 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 16:38:06.098511 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093240 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 16:38:06.098511 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093243 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 16:38:06.098511 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093246 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 16:38:06.098511 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093248 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 16:38:06.098511 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093251 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 16:38:06.098511 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093253 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 16:38:06.098511 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093256 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 06 16:38:06.098511 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093258 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 16:38:06.098511 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093261 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 16:38:06.098511 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093263 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 16:38:06.098511 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093265 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 16:38:06.098990 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093269 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 16:38:06.098990 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093272 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 16:38:06.098990 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093275 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 16:38:06.098990 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093278 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 16:38:06.098990 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093280 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 16:38:06.098990 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093283 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 16:38:06.098990 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093286 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 16:38:06.098990 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093288 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 16:38:06.098990 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093291 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 16:38:06.098990 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093293 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 16:38:06.098990 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093296 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 16:38:06.098990 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093298 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 16:38:06.098990 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093301 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 16:38:06.098990 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093303 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 16:38:06.098990 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093306 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 16:38:06.098990 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093309 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 16:38:06.098990 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093311 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 16:38:06.098990 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093314 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 16:38:06.098990 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093316 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 16:38:06.098990 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093319 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 16:38:06.099545 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093322 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 16:38:06.099545 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093324 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 16:38:06.099545 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093326 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 16:38:06.099545 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093329 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 16:38:06.099545 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093331 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 16:38:06.099545 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093334 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 16:38:06.099545 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:06.093336 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 16:38:06.099545 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.093341 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 16:38:06.099545 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.093447 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 16:38:06.099545 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.097078 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 16:38:06.099545 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.098415 2580 server.go:1019] "Starting client certificate rotation" Jul 06 16:38:06.099545 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.098508 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 16:38:06.099545 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.098569 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 16:38:06.100972 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.100959 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 16:38:06.104330 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.104311 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 16:38:06.114393 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.114376 2580 log.go:25] "Validated CRI v1 runtime API" Jul 06 16:38:06.117813 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.117791 2580 log.go:25] "Validated CRI v1 image API" Jul 06 16:38:06.119184 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.119165 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 16:38:06.121579 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.121561 2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 84701fed-3a1f-42e3-8089-9e72ebfac196:/dev/nvme0n1p4 b772a2ae-c778-4c24-9a12-d2b96c2f44d8:/dev/nvme0n1p3] Jul 06 16:38:06.121647 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.121578 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 16:38:06.127790 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.127687 2580 manager.go:216] Machine: {Timestamp:2026-07-06 16:38:06.126493709 +0000 UTC m=+0.304352812 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102766 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec233a3502e0d9264612b684d5cf4fec SystemUUID:ec233a35-02e0-d926-4612-b684d5cf4fec BootID:41cbdc66-504a-4afc-bed6-f025742ad51b Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:53:00:4f:93:af Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:53:00:4f:93:af Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2a:19:54:57:de:00 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 16:38:06.127790 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.127783 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 16:38:06.127914 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.127854 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 16:38:06.129520 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.129497 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 16:38:06.129654 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.129522 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-40.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 16:38:06.130252 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.130243 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 16:38:06.130282 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.130255 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 16:38:06.130282 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.130263 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 16:38:06.131545 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.131531 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 16:38:06.133043 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.133032 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 16:38:06.137027 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.137014 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 06 16:38:06.137137 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.137128 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 16:38:06.137177 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.137156 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 06 16:38:06.137177 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.137165 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 16:38:06.137231 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.137178 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 16:38:06.137231 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.137187 2580 kubelet.go:398] "Adding apiserver pod source" Jul 06 16:38:06.137231 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.137195 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 16:38:06.137556 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.137542 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 16:38:06.137613 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.137563 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 16:38:06.139414 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.139394 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 16:38:06.141420 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.141406 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 16:38:06.141464 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.141438 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 16:38:06.142593 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.142581 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 16:38:06.142626 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.142601 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 16:38:06.142626 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.142609 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 16:38:06.142626 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.142615 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 16:38:06.142626 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.142624 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 16:38:06.142732 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.142629 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 16:38:06.142732 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.142638 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 16:38:06.142732 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.142644 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 16:38:06.142732 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.142650 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 16:38:06.142732 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.142656 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 16:38:06.142732 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.142664 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 16:38:06.142732 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.142670 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 16:38:06.143416 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.143403 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 16:38:06.143416 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.143414 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 16:38:06.143960 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.143942 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 16:38:06.146480 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.146455 2580 server.go:1268] "Started kubelet" Jul 06 16:38:06.146611 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.146585 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 16:38:06.146662 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.146599 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 16:38:06.146703 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.146676 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 16:38:06.147388 ip-10-0-135-40 systemd[1]: Started Kubernetes Kubelet. Jul 06 16:38:06.148583 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.148523 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 16:38:06.149715 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.149696 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 06 16:38:06.150812 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.150796 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-40.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 16:38:06.150910 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.150872 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-40.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 16:38:06.150910 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.150872 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 16:38:06.152234 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.152218 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 16:38:06.156452 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.155516 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-40.ec2.internal.18bfc0bad5dc21c5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-40.ec2.internal,UID:ip-10-0-135-40.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-40.ec2.internal,},FirstTimestamp:2026-07-06 16:38:06.146429381 +0000 UTC m=+0.324288550,LastTimestamp:2026-07-06 16:38:06.146429381 +0000 UTC m=+0.324288550,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-40.ec2.internal,}" Jul 06 16:38:06.156781 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.156766 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 16:38:06.156864 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.156808 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 16:38:06.157143 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.157110 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found" Jul 06 16:38:06.157972 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.157539 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 16:38:06.157972 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.157540 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 16:38:06.157972 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.157568 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 16:38:06.157972 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.157630 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 06 16:38:06.157972 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.157640 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 06 16:38:06.157972 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.157826 2580 factory.go:55] Registering systemd factory Jul 06 16:38:06.157972 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.157845 2580 factory.go:223] Registration of the systemd container factory successfully Jul 06 16:38:06.158420 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.158198 2580 factory.go:153] Registering CRI-O factory Jul 06 16:38:06.158420 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.158209 2580 factory.go:223] Registration of the crio container factory successfully Jul 06 16:38:06.158420 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.158294 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 16:38:06.158789 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.158595 2580 factory.go:103] Registering Raw factory Jul 06 16:38:06.158789 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.158616 2580 manager.go:1196] Started watching for new ooms in manager Jul 06 16:38:06.159235 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.159220 2580 manager.go:349] Starting recovery of all containers Jul 06 16:38:06.166487 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.166441 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-40.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 16:38:06.166941 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.166919 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 16:38:06.171452 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.171426 2580 manager.go:354] Recovery completed Jul 06 16:38:06.175563 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.175551 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 16:38:06.177433 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.177410 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientMemory" Jul 06 16:38:06.177518 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.177445 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 16:38:06.177518 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.177455 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientPID" Jul 06 16:38:06.177941 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.177923 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 16:38:06.177941 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.177939 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 16:38:06.178078 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.177939 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 16:38:06.178078 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.177992 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 06 16:38:06.179086 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.179070 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientMemory" Jul 06 16:38:06.179151 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.179102 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 16:38:06.179151 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.179116 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientPID" Jul 06 16:38:06.180130 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.180112 2580 policy_none.go:49] "None policy: Start" Jul 06 16:38:06.180130 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.180128 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 16:38:06.180288 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.180141 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 16:38:06.180397 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.180320 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-40.ec2.internal.18bfc0bad7b52e35 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-40.ec2.internal,UID:ip-10-0-135-40.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-40.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-40.ec2.internal,},FirstTimestamp:2026-07-06 16:38:06.177431093 +0000 UTC m=+0.355290201,LastTimestamp:2026-07-06 16:38:06.177431093 +0000 UTC m=+0.355290201,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-40.ec2.internal,}" Jul 06 16:38:06.182202 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.182190 2580 policy_none.go:47] "Start" Jul 06 16:38:06.194834 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.194772 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-40.ec2.internal.18bfc0bad7b5796c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-40.ec2.internal,UID:ip-10-0-135-40.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-40.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-40.ec2.internal,},FirstTimestamp:2026-07-06 16:38:06.177450348 +0000 UTC m=+0.355309452,LastTimestamp:2026-07-06 16:38:06.177450348 +0000 UTC m=+0.355309452,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-40.ec2.internal,}" Jul 06 16:38:06.206813 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.206657 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-40.ec2.internal.18bfc0bad7b59d15 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-40.ec2.internal,UID:ip-10-0-135-40.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-40.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-40.ec2.internal,},FirstTimestamp:2026-07-06 16:38:06.177459477 +0000 UTC m=+0.355318580,LastTimestamp:2026-07-06 16:38:06.177459477 +0000 UTC m=+0.355318580,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-40.ec2.internal,}" Jul 06 16:38:06.209941 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.209919 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ssjk8" Jul 06 16:38:06.216673 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.216606 2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-40.ec2.internal.18bfc0bad7b52e35\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-40.ec2.internal.18bfc0bad7b52e35 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-40.ec2.internal,UID:ip-10-0-135-40.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-40.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-40.ec2.internal,},FirstTimestamp:2026-07-06 16:38:06.177431093 +0000 UTC m=+0.355290201,LastTimestamp:2026-07-06 16:38:06.179086738 +0000 UTC m=+0.356945846,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-40.ec2.internal,}" Jul 06 16:38:06.224162 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.224148 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ssjk8" Jul 06 16:38:06.228336 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.228280 2580 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-135-40.ec2.internal.18bfc0bad7b5796c\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-40.ec2.internal.18bfc0bad7b5796c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-40.ec2.internal,UID:ip-10-0-135-40.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-40.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-40.ec2.internal,},FirstTimestamp:2026-07-06 16:38:06.177450348 +0000 UTC m=+0.355309452,LastTimestamp:2026-07-06 16:38:06.179108169 +0000 UTC m=+0.356967273,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-40.ec2.internal,}" Jul 06 16:38:06.229050 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.229039 2580 manager.go:337] "Starting Device Plugin manager" Jul 06 16:38:06.229090 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.229076 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 16:38:06.229090 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.229090 2580 server.go:87] "Starting device plugin registration server" Jul 06 16:38:06.229321 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.229310 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 16:38:06.229353 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.229323 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 16:38:06.229441 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.229420 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 16:38:06.229551 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.229509 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 16:38:06.229551 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.229516 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 16:38:06.229949 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.229932 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 16:38:06.230006 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.229975 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-40.ec2.internal\" not found" Jul 06 16:38:06.256949 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.256928 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 16:38:06.264605 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.258083 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 16:38:06.264605 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.258116 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 16:38:06.264605 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.258135 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 16:38:06.264605 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.258167 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 16:38:06.264605 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.261014 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 16:38:06.329828 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.329760 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 16:38:06.330681 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.330665 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientMemory" Jul 06 16:38:06.330761 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.330693 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 16:38:06.330761 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.330702 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientPID" Jul 06 16:38:06.330761 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.330725 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-40.ec2.internal" Jul 06 16:38:06.341558 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.341541 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-40.ec2.internal" Jul 06 16:38:06.341614 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.341566 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-40.ec2.internal\": node \"ip-10-0-135-40.ec2.internal\" not found" Jul 06 16:38:06.358619 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.358597 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal"] Jul 06 16:38:06.358696 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.358654 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 16:38:06.359964 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.359948 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientMemory" Jul 06 16:38:06.360062 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.359974 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 16:38:06.360062 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.359987 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientPID" Jul 06 16:38:06.362326 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.362312 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 16:38:06.362424 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.362410 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal" Jul 06 16:38:06.362424 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.362437 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 16:38:06.362958 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.362943 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientMemory" Jul 06 16:38:06.363025 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.362966 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientMemory" Jul 06 16:38:06.363025 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.362986 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 16:38:06.363025 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.362996 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientPID" Jul 06 16:38:06.363114 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.362970 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 16:38:06.363114 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.363051 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientPID" Jul 06 16:38:06.365339 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.365323 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal" Jul 06 16:38:06.365408 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.365356 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 16:38:06.365982 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.365965 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientMemory" Jul 06 16:38:06.366102 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.365992 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 16:38:06.366102 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.366008 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientPID" Jul 06 16:38:06.384014 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.383993 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-40.ec2.internal\" not found" node="ip-10-0-135-40.ec2.internal" Jul 06 16:38:06.388404 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.388389 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-40.ec2.internal\" not found" node="ip-10-0-135-40.ec2.internal" Jul 06 16:38:06.393831 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.393816 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found" Jul 06 16:38:06.458837 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.458807 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e07cd2e8cb8f8817f3a6b417cfd6e9f6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal\" (UID: \"e07cd2e8cb8f8817f3a6b417cfd6e9f6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal" Jul 06 16:38:06.458962 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.458839 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e07cd2e8cb8f8817f3a6b417cfd6e9f6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal\" (UID: \"e07cd2e8cb8f8817f3a6b417cfd6e9f6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal" Jul 06 16:38:06.458962 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.458858 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b5e29f9f4d849ce3821c6ee834e1493a-config\") pod \"kube-apiserver-proxy-ip-10-0-135-40.ec2.internal\" (UID: \"b5e29f9f4d849ce3821c6ee834e1493a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal" Jul 06 16:38:06.495784 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.494101 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found" Jul 06 16:38:06.559481 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.559445 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e07cd2e8cb8f8817f3a6b417cfd6e9f6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal\" (UID: \"e07cd2e8cb8f8817f3a6b417cfd6e9f6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal" Jul 06 16:38:06.559554 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.559489 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e07cd2e8cb8f8817f3a6b417cfd6e9f6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal\" (UID: \"e07cd2e8cb8f8817f3a6b417cfd6e9f6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal" Jul 06 16:38:06.559554 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.559508 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b5e29f9f4d849ce3821c6ee834e1493a-config\") pod \"kube-apiserver-proxy-ip-10-0-135-40.ec2.internal\" (UID: \"b5e29f9f4d849ce3821c6ee834e1493a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal" Jul 06 16:38:06.559554 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.559542 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b5e29f9f4d849ce3821c6ee834e1493a-config\") pod \"kube-apiserver-proxy-ip-10-0-135-40.ec2.internal\" (UID: \"b5e29f9f4d849ce3821c6ee834e1493a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal" Jul 06 16:38:06.559554 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.559549 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e07cd2e8cb8f8817f3a6b417cfd6e9f6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal\" (UID: \"e07cd2e8cb8f8817f3a6b417cfd6e9f6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal" Jul 06 16:38:06.559676 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.559555 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e07cd2e8cb8f8817f3a6b417cfd6e9f6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal\" (UID: \"e07cd2e8cb8f8817f3a6b417cfd6e9f6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal" Jul 06 16:38:06.595441 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.595391 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found" Jul 06 16:38:06.688673 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.688641 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal" Jul 06 16:38:06.692371 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:06.692353 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal" Jul 06 16:38:06.696363 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.696347 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found" Jul 06 16:38:06.796575 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.796540 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found" Jul 06 16:38:06.898636 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:06.898560 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found" Jul 06 16:38:07.000716 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:07.000676 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found" Jul 06 16:38:07.099643 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:07.099614 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 16:38:07.100051 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:07.099764 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 16:38:07.100790 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:07.100773 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found" Jul 06 16:38:07.201838 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:07.201773 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found" Jul 06 16:38:07.226809 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:07.226775 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 16:33:06 +0000 UTC" deadline="2027-12-17 23:14:54.512561486 +0000 UTC" Jul 06 16:38:07.226877 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:07.226811 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12702h36m47.285759272s" Jul 06 16:38:07.287785 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:07.287761 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 16:38:07.289129 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:07.289115 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientMemory" Jul 06 16:38:07.289210 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:07.289147 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 16:38:07.289210 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:07.289162 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeHasSufficientPID" Jul 06 16:38:07.302782 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:07.302760 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found" Jul 06 16:38:07.335995 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:07.335974 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 16:38:07.402953 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:07.402923 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found" Jul 06 16:38:07.504954 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:07.504889 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-40.ec2.internal\" not found" Jul 06 16:38:07.538364 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:07.538344 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 16:38:07.544197 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:07.544170 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5e29f9f4d849ce3821c6ee834e1493a.slice/crio-a20ecd09f54991f8b18d97dd6f421986ac282dc8de3172a63cf458d021fe3a73 WatchSource:0}: Error finding container a20ecd09f54991f8b18d97dd6f421986ac282dc8de3172a63cf458d021fe3a73: Status 404 returned error can't find the container with id a20ecd09f54991f8b18d97dd6f421986ac282dc8de3172a63cf458d021fe3a73 Jul 06 16:38:07.550498 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:07.550470 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 16:38:07.556536 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:07.556513 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode07cd2e8cb8f8817f3a6b417cfd6e9f6.slice/crio-5f05aea8903bef1f6d53fee8c90fd7fdaa1e8b897782070189de48d48421acaf WatchSource:0}: Error finding container 5f05aea8903bef1f6d53fee8c90fd7fdaa1e8b897782070189de48d48421acaf: Status 404 returned error can't find the container with id 5f05aea8903bef1f6d53fee8c90fd7fdaa1e8b897782070189de48d48421acaf Jul 06 16:38:07.557217 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:07.557190 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal" Jul 06 16:38:07.592813 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:07.592789 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 16:38:07.593815 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:07.593800 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal" Jul 06 16:38:07.621062 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:07.621045 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 16:38:07.743586 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:07.743563 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 16:38:08.140010 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.139971 2580 apiserver.go:52] "Watching apiserver" Jul 06 16:38:08.145072 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.145048 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 16:38:08.145456 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.145433 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-mt8jg","openshift-dns/node-resolver-q4flx","openshift-image-registry/node-ca-zw578","openshift-multus/multus-k6c8s","openshift-multus/network-metrics-daemon-5m692","openshift-network-operator/iptables-alerter-nphrg","kube-system/global-pull-secret-syncer-2m87r","kube-system/konnectivity-agent-xqf97","kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal","openshift-multus/multus-additional-cni-plugins-44nqc","openshift-network-diagnostics/network-check-target-xzfns","openshift-ovn-kubernetes/ovnkube-node-jrb4n"] Jul 06 16:38:08.148515 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.148495 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.150709 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.150686 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q4flx" Jul 06 16:38:08.151539 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.151521 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 16:38:08.151863 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.151849 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-td2wd\"" Jul 06 16:38:08.151957 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.151941 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 16:38:08.152005 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.151988 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 16:38:08.152598 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.152583 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 16:38:08.152957 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.152940 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 16:38:08.153042 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.153026 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zw578" Jul 06 16:38:08.153191 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.153167 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 16:38:08.153335 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.153317 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 16:38:08.153561 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.153547 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5rr97\"" Jul 06 16:38:08.155922 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.155738 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.157943 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.157920 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 16:38:08.158588 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.158561 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:08.158718 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:08.158638 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m692" podUID="3f867a4a-dbf4-46a7-a79b-c30d916fa0f8" Jul 06 16:38:08.159285 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.159268 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 16:38:08.159408 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.159379 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 16:38:08.159530 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.159437 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 16:38:08.159530 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.159437 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-plr8h\"" Jul 06 16:38:08.159530 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.159383 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 16:38:08.159530 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.159447 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lvln9\"" Jul 06 16:38:08.160957 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.160938 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nphrg" Jul 06 16:38:08.163501 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.163468 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.165165 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.165146 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 16:38:08.165512 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.165481 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 16:38:08.165907 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.165889 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:08.166002 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:08.165974 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzfns" podUID="52ebf132-7501-49f4-8615-28020de19a6f" Jul 06 16:38:08.167758 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.167740 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-w69mj\"" Jul 06 16:38:08.167758 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.167740 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 16:38:08.167922 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.167897 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 16:38:08.167922 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.167901 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 16:38:08.168124 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.168107 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 16:38:08.168199 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.168158 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 16:38:08.168199 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.168190 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 16:38:08.168305 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.168243 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 16:38:08.168305 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.168260 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mmvdt\"" Jul 06 16:38:08.168432 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.168414 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:08.168545 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:08.168463 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2m87r" podUID="0f5cbd3c-3a54-4441-bea2-2643a5912979" Jul 06 16:38:08.170993 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.170964 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.172116 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172096 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-cnibin\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.172197 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172124 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqrdp\" (UniqueName: \"kubernetes.io/projected/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-kube-api-access-jqrdp\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.172197 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172143 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkdf4\" (UniqueName: \"kubernetes.io/projected/cf835c0c-f2ea-416f-a27f-619d7eeaf9c6-kube-api-access-dkdf4\") pod \"node-resolver-q4flx\" (UID: \"cf835c0c-f2ea-416f-a27f-619d7eeaf9c6\") " pod="openshift-dns/node-resolver-q4flx" Jul 06 16:38:08.172197 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172159 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbbwg\" (UniqueName: \"kubernetes.io/projected/206b0067-d25e-40fd-ab2d-5a834df0c969-kube-api-access-sbbwg\") pod \"node-ca-zw578\" (UID: \"206b0067-d25e-40fd-ab2d-5a834df0c969\") " pod="openshift-image-registry/node-ca-zw578" Jul 06 16:38:08.172197 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172175 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-host-var-lib-cni-bin\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.172377 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172227 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-host-var-lib-kubelet\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.172377 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172257 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-system-cni-dir\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.172377 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172283 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-host-run-k8s-cni-cncf-io\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.172377 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172303 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-host-run-netns\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.172377 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172318 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-hostroot\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.172377 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172340 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.172377 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172366 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-multus-conf-dir\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.172794 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172408 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.172794 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172492 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf835c0c-f2ea-416f-a27f-619d7eeaf9c6-tmp-dir\") pod \"node-resolver-q4flx\" (UID: \"cf835c0c-f2ea-416f-a27f-619d7eeaf9c6\") " pod="openshift-dns/node-resolver-q4flx" Jul 06 16:38:08.172794 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172527 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/206b0067-d25e-40fd-ab2d-5a834df0c969-host\") pod \"node-ca-zw578\" (UID: \"206b0067-d25e-40fd-ab2d-5a834df0c969\") " pod="openshift-image-registry/node-ca-zw578" Jul 06 16:38:08.172794 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172554 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-multus-cni-dir\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.172794 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172579 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-etc-kubernetes\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.172794 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172602 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs\") pod \"network-metrics-daemon-5m692\" (UID: \"3f867a4a-dbf4-46a7-a79b-c30d916fa0f8\") " pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:08.172794 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172625 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-os-release\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.172794 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172652 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.172794 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172677 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cf835c0c-f2ea-416f-a27f-619d7eeaf9c6-hosts-file\") pod \"node-resolver-q4flx\" (UID: \"cf835c0c-f2ea-416f-a27f-619d7eeaf9c6\") " pod="openshift-dns/node-resolver-q4flx" Jul 06 16:38:08.172794 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172710 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/206b0067-d25e-40fd-ab2d-5a834df0c969-serviceca\") pod \"node-ca-zw578\" (UID: \"206b0067-d25e-40fd-ab2d-5a834df0c969\") " pod="openshift-image-registry/node-ca-zw578" Jul 06 16:38:08.172794 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172739 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwsbv\" (UniqueName: \"kubernetes.io/projected/e07c98e1-5907-417d-91c3-ffd8a87fab85-kube-api-access-gwsbv\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.172794 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172767 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-system-cni-dir\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.172794 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172791 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-host-var-lib-cni-multus\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.173224 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172816 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-cni-binary-copy\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.173224 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172840 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-cnibin\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.173224 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172866 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-os-release\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.173224 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172889 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e07c98e1-5907-417d-91c3-ffd8a87fab85-cni-binary-copy\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.173224 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172914 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-multus-socket-dir-parent\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.173224 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172938 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e07c98e1-5907-417d-91c3-ffd8a87fab85-multus-daemon-config\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.173224 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172972 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7bxq\" (UniqueName: \"kubernetes.io/projected/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-kube-api-access-t7bxq\") pod \"network-metrics-daemon-5m692\" (UID: \"3f867a4a-dbf4-46a7-a79b-c30d916fa0f8\") " pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:08.173224 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.172996 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-host-run-multus-certs\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.173653 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.173635 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xqf97" Jul 06 16:38:08.174649 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.174632 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 16:38:08.174740 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.174638 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 16:38:08.174882 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.174862 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-c6pvq\"" Jul 06 16:38:08.175691 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.175653 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 16:38:08.175791 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.175778 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 16:38:08.176074 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.176060 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.176184 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.176168 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-k2xmx\"" Jul 06 16:38:08.176545 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.176527 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 16:38:08.177956 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.177941 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 16:38:08.178453 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.178438 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mvpjt\"" Jul 06 16:38:08.179219 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.179201 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 16:38:08.196944 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.196921 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 16:38:08.211652 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.211629 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fdhtb" Jul 06 16:38:08.227260 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.227238 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fdhtb" Jul 06 16:38:08.259016 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.258988 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 16:38:08.263912 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.263864 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal" event={"ID":"e07cd2e8cb8f8817f3a6b417cfd6e9f6","Type":"ContainerStarted","Data":"5f05aea8903bef1f6d53fee8c90fd7fdaa1e8b897782070189de48d48421acaf"} Jul 06 16:38:08.265309 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.265261 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal" event={"ID":"b5e29f9f4d849ce3821c6ee834e1493a","Type":"ContainerStarted","Data":"a20ecd09f54991f8b18d97dd6f421986ac282dc8de3172a63cf458d021fe3a73"} Jul 06 16:38:08.273428 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.273404 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-host-var-lib-cni-multus\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.273555 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.273446 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ca56b80c-c9df-4f01-b5b2-134faaedab44-iptables-alerter-script\") pod \"iptables-alerter-nphrg\" (UID: \"ca56b80c-c9df-4f01-b5b2-134faaedab44\") " pod="openshift-network-operator/iptables-alerter-nphrg" Jul 06 16:38:08.273555 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.273473 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-log-socket\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.273555 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.273514 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-cni-binary-copy\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.273555 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.273542 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-cnibin\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.273774 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.273548 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-host-var-lib-cni-multus\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.273774 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.273607 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e07c98e1-5907-417d-91c3-ffd8a87fab85-cni-binary-copy\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.273774 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.273648 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-var-lib-kubelet\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.273774 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.273663 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-cnibin\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.274028 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274005 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7bxq\" (UniqueName: \"kubernetes.io/projected/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-kube-api-access-t7bxq\") pod \"network-metrics-daemon-5m692\" (UID: \"3f867a4a-dbf4-46a7-a79b-c30d916fa0f8\") " pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:08.274090 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274050 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-host-cni-bin\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.274145 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274115 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkkpb\" (UniqueName: \"kubernetes.io/projected/84f4b2fb-a67b-4443-aba6-a54f0f17c751-kube-api-access-nkkpb\") pod \"aws-ebs-csi-driver-node-wkb7v\" (UID: \"84f4b2fb-a67b-4443-aba6-a54f0f17c751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.274213 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274157 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-lib-modules\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.274274 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274218 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-cnibin\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.274274 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274256 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkdf4\" (UniqueName: \"kubernetes.io/projected/cf835c0c-f2ea-416f-a27f-619d7eeaf9c6-kube-api-access-dkdf4\") pod \"node-resolver-q4flx\" (UID: \"cf835c0c-f2ea-416f-a27f-619d7eeaf9c6\") " pod="openshift-dns/node-resolver-q4flx" Jul 06 16:38:08.274380 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274283 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-host-var-lib-cni-bin\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.274380 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274287 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-cni-binary-copy\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.274380 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274330 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-host-kubelet\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.274380 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274356 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-cnibin\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.274684 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274401 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-host-run-ovn-kubernetes\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.274684 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274407 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e07c98e1-5907-417d-91c3-ffd8a87fab85-cni-binary-copy\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.274684 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274431 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czwvb\" (UniqueName: \"kubernetes.io/projected/478a85da-6e22-4a45-8ab8-bd99215ec846-kube-api-access-czwvb\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.274684 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274465 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-host-var-lib-cni-bin\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.274684 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274533 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0f5cbd3c-3a54-4441-bea2-2643a5912979-dbus\") pod \"global-pull-secret-syncer-2m87r\" (UID: \"0f5cbd3c-3a54-4441-bea2-2643a5912979\") " pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:08.274684 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274564 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-host-run-k8s-cni-cncf-io\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.274684 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274591 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-host-run-netns\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.274684 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274632 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-host-run-netns\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.274684 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274667 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-hostroot\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.275137 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274695 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-host-run-netns\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.275137 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274723 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-host-cni-netd\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.275137 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274730 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-hostroot\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.275137 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274764 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/478a85da-6e22-4a45-8ab8-bd99215ec846-env-overrides\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.275137 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274782 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-host-run-k8s-cni-cncf-io\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.275137 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274791 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret\") pod \"global-pull-secret-syncer-2m87r\" (UID: \"0f5cbd3c-3a54-4441-bea2-2643a5912979\") " pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:08.275137 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274834 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-etc-kubernetes\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.275137 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274868 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.275137 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274896 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-host-slash\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.275137 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274936 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.275137 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.274963 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf835c0c-f2ea-416f-a27f-619d7eeaf9c6-tmp-dir\") pod \"node-resolver-q4flx\" (UID: \"cf835c0c-f2ea-416f-a27f-619d7eeaf9c6\") " pod="openshift-dns/node-resolver-q4flx" Jul 06 16:38:08.275137 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275007 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/206b0067-d25e-40fd-ab2d-5a834df0c969-host\") pod \"node-ca-zw578\" (UID: \"206b0067-d25e-40fd-ab2d-5a834df0c969\") " pod="openshift-image-registry/node-ca-zw578" Jul 06 16:38:08.275137 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275031 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-multus-cni-dir\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.275137 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275054 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-etc-kubernetes\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.275137 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275081 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mx2tp\" (UniqueName: \"kubernetes.io/projected/ca56b80c-c9df-4f01-b5b2-134faaedab44-kube-api-access-mx2tp\") pod \"iptables-alerter-nphrg\" (UID: \"ca56b80c-c9df-4f01-b5b2-134faaedab44\") " pod="openshift-network-operator/iptables-alerter-nphrg" Jul 06 16:38:08.275137 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275108 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-systemd-units\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.275137 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275136 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-os-release\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.276024 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275147 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-tuning-conf-dir\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.276024 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275161 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.276024 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275186 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/206b0067-d25e-40fd-ab2d-5a834df0c969-serviceca\") pod \"node-ca-zw578\" (UID: \"206b0067-d25e-40fd-ab2d-5a834df0c969\") " pod="openshift-image-registry/node-ca-zw578" Jul 06 16:38:08.276024 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275210 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/478a85da-6e22-4a45-8ab8-bd99215ec846-ovn-node-metrics-cert\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.276024 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275237 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0f5cbd3c-3a54-4441-bea2-2643a5912979-kubelet-config\") pod \"global-pull-secret-syncer-2m87r\" (UID: \"0f5cbd3c-3a54-4441-bea2-2643a5912979\") " pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:08.276024 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275260 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-run\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.276024 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275286 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-sys\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.276024 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275311 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/66cb350d-1b5e-4f99-8cee-0a68c4600341-etc-tuned\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.276024 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275339 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-system-cni-dir\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.276024 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275364 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/84f4b2fb-a67b-4443-aba6-a54f0f17c751-etc-selinux\") pod \"aws-ebs-csi-driver-node-wkb7v\" (UID: \"84f4b2fb-a67b-4443-aba6-a54f0f17c751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.276024 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275384 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-multus-cni-dir\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.276024 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275389 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-os-release\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.276024 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275413 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.276024 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275434 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-multus-socket-dir-parent\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.276024 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275455 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-os-release\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.276024 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275461 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e07c98e1-5907-417d-91c3-ffd8a87fab85-multus-daemon-config\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.276024 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275492 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-node-log\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.276965 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275505 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-etc-kubernetes\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.276965 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275537 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-host\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.276965 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275565 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7ncp\" (UniqueName: \"kubernetes.io/projected/66cb350d-1b5e-4f99-8cee-0a68c4600341-kube-api-access-c7ncp\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.276965 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275597 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-host-run-multus-certs\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.276965 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275597 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-os-release\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.276965 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275624 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca56b80c-c9df-4f01-b5b2-134faaedab44-host-slash\") pod \"iptables-alerter-nphrg\" (UID: \"ca56b80c-c9df-4f01-b5b2-134faaedab44\") " pod="openshift-network-operator/iptables-alerter-nphrg" Jul 06 16:38:08.276965 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275649 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-etc-openvswitch\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.276965 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275675 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/84f4b2fb-a67b-4443-aba6-a54f0f17c751-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wkb7v\" (UID: \"84f4b2fb-a67b-4443-aba6-a54f0f17c751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.276965 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275700 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/66c85ad6-1024-423c-b7bc-2c17115326c1-agent-certs\") pod \"konnectivity-agent-xqf97\" (UID: \"66c85ad6-1024-423c-b7bc-2c17115326c1\") " pod="kube-system/konnectivity-agent-xqf97" Jul 06 16:38:08.276965 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275726 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-etc-sysconfig\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.276965 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275741 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf835c0c-f2ea-416f-a27f-619d7eeaf9c6-tmp-dir\") pod \"node-resolver-q4flx\" (UID: \"cf835c0c-f2ea-416f-a27f-619d7eeaf9c6\") " pod="openshift-dns/node-resolver-q4flx" Jul 06 16:38:08.276965 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275754 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqrdp\" (UniqueName: \"kubernetes.io/projected/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-kube-api-access-jqrdp\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.276965 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275782 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbbwg\" (UniqueName: \"kubernetes.io/projected/206b0067-d25e-40fd-ab2d-5a834df0c969-kube-api-access-sbbwg\") pod \"node-ca-zw578\" (UID: \"206b0067-d25e-40fd-ab2d-5a834df0c969\") " pod="openshift-image-registry/node-ca-zw578" Jul 06 16:38:08.276965 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275813 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/206b0067-d25e-40fd-ab2d-5a834df0c969-host\") pod \"node-ca-zw578\" (UID: \"206b0067-d25e-40fd-ab2d-5a834df0c969\") " pod="openshift-image-registry/node-ca-zw578" Jul 06 16:38:08.276965 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275825 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-host-var-lib-kubelet\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.276965 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275855 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/478a85da-6e22-4a45-8ab8-bd99215ec846-ovnkube-script-lib\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.276965 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275880 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/84f4b2fb-a67b-4443-aba6-a54f0f17c751-socket-dir\") pod \"aws-ebs-csi-driver-node-wkb7v\" (UID: \"84f4b2fb-a67b-4443-aba6-a54f0f17c751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.277787 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275917 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/84f4b2fb-a67b-4443-aba6-a54f0f17c751-device-dir\") pod \"aws-ebs-csi-driver-node-wkb7v\" (UID: \"84f4b2fb-a67b-4443-aba6-a54f0f17c751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.277787 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275944 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-etc-systemd\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.277787 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275868 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-multus-socket-dir-parent\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.277787 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.275975 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-system-cni-dir\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.277787 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276004 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.277787 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276033 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/66c85ad6-1024-423c-b7bc-2c17115326c1-konnectivity-ca\") pod \"konnectivity-agent-xqf97\" (UID: \"66c85ad6-1024-423c-b7bc-2c17115326c1\") " pod="kube-system/konnectivity-agent-xqf97" Jul 06 16:38:08.277787 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276059 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-multus-conf-dir\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.277787 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276087 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-run-openvswitch\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.277787 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276111 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-run-ovn\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.277787 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276116 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.277787 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276140 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/84f4b2fb-a67b-4443-aba6-a54f0f17c751-registration-dir\") pod \"aws-ebs-csi-driver-node-wkb7v\" (UID: \"84f4b2fb-a67b-4443-aba6-a54f0f17c751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.277787 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276165 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-etc-modprobe-d\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.277787 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276190 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-run-systemd\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.277787 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276215 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/478a85da-6e22-4a45-8ab8-bd99215ec846-ovnkube-config\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.277787 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276251 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-var-lib-openvswitch\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.277787 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276279 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-etc-sysctl-d\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.278455 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276308 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs\") pod \"network-metrics-daemon-5m692\" (UID: \"3f867a4a-dbf4-46a7-a79b-c30d916fa0f8\") " pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:08.278455 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276337 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cf835c0c-f2ea-416f-a27f-619d7eeaf9c6-hosts-file\") pod \"node-resolver-q4flx\" (UID: \"cf835c0c-f2ea-416f-a27f-619d7eeaf9c6\") " pod="openshift-dns/node-resolver-q4flx" Jul 06 16:38:08.278455 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276415 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwsbv\" (UniqueName: \"kubernetes.io/projected/e07c98e1-5907-417d-91c3-ffd8a87fab85-kube-api-access-gwsbv\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.278455 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276422 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e07c98e1-5907-417d-91c3-ffd8a87fab85-multus-daemon-config\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.278455 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276479 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9bd9\" (UniqueName: \"kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9\") pod \"network-check-target-xzfns\" (UID: \"52ebf132-7501-49f4-8615-28020de19a6f\") " pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:08.278455 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276507 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/84f4b2fb-a67b-4443-aba6-a54f0f17c751-sys-fs\") pod \"aws-ebs-csi-driver-node-wkb7v\" (UID: \"84f4b2fb-a67b-4443-aba6-a54f0f17c751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.278455 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276511 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-host-run-multus-certs\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.278455 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276576 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/206b0067-d25e-40fd-ab2d-5a834df0c969-serviceca\") pod \"node-ca-zw578\" (UID: \"206b0067-d25e-40fd-ab2d-5a834df0c969\") " pod="openshift-image-registry/node-ca-zw578" Jul 06 16:38:08.278455 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276541 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-etc-sysctl-conf\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.278455 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276640 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66cb350d-1b5e-4f99-8cee-0a68c4600341-tmp\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.278455 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276763 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-system-cni-dir\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.278455 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276768 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-system-cni-dir\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.278455 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276816 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-multus-conf-dir\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.278455 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.276877 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e07c98e1-5907-417d-91c3-ffd8a87fab85-host-var-lib-kubelet\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.278455 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.277004 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cf835c0c-f2ea-416f-a27f-619d7eeaf9c6-hosts-file\") pod \"node-resolver-q4flx\" (UID: \"cf835c0c-f2ea-416f-a27f-619d7eeaf9c6\") " pod="openshift-dns/node-resolver-q4flx" Jul 06 16:38:08.278455 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:08.277079 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:38:08.278455 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:08.277147 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs podName:3f867a4a-dbf4-46a7-a79b-c30d916fa0f8 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:08.777126582 +0000 UTC m=+2.954985676 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs") pod "network-metrics-daemon-5m692" (UID: "3f867a4a-dbf4-46a7-a79b-c30d916fa0f8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:38:08.288845 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.288779 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 16:38:08.293233 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.293207 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7bxq\" (UniqueName: \"kubernetes.io/projected/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-kube-api-access-t7bxq\") pod \"network-metrics-daemon-5m692\" (UID: \"3f867a4a-dbf4-46a7-a79b-c30d916fa0f8\") " pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:08.293782 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.293750 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkdf4\" (UniqueName: \"kubernetes.io/projected/cf835c0c-f2ea-416f-a27f-619d7eeaf9c6-kube-api-access-dkdf4\") pod \"node-resolver-q4flx\" (UID: \"cf835c0c-f2ea-416f-a27f-619d7eeaf9c6\") " pod="openshift-dns/node-resolver-q4flx" Jul 06 16:38:08.294339 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.294300 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqrdp\" (UniqueName: \"kubernetes.io/projected/6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb-kube-api-access-jqrdp\") pod \"multus-additional-cni-plugins-44nqc\" (UID: \"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb\") " pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.294813 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.294795 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwsbv\" (UniqueName: \"kubernetes.io/projected/e07c98e1-5907-417d-91c3-ffd8a87fab85-kube-api-access-gwsbv\") pod \"multus-k6c8s\" (UID: \"e07c98e1-5907-417d-91c3-ffd8a87fab85\") " pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.296232 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.296212 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbbwg\" (UniqueName: \"kubernetes.io/projected/206b0067-d25e-40fd-ab2d-5a834df0c969-kube-api-access-sbbwg\") pod \"node-ca-zw578\" (UID: \"206b0067-d25e-40fd-ab2d-5a834df0c969\") " pod="openshift-image-registry/node-ca-zw578" Jul 06 16:38:08.377408 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377374 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-node-log\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.377408 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377413 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-host\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.377666 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377441 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7ncp\" (UniqueName: \"kubernetes.io/projected/66cb350d-1b5e-4f99-8cee-0a68c4600341-kube-api-access-c7ncp\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.377666 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377508 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-node-log\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.377666 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377512 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca56b80c-c9df-4f01-b5b2-134faaedab44-host-slash\") pod \"iptables-alerter-nphrg\" (UID: \"ca56b80c-c9df-4f01-b5b2-134faaedab44\") " pod="openshift-network-operator/iptables-alerter-nphrg" Jul 06 16:38:08.377666 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377539 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-host\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.377666 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377560 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ca56b80c-c9df-4f01-b5b2-134faaedab44-host-slash\") pod \"iptables-alerter-nphrg\" (UID: \"ca56b80c-c9df-4f01-b5b2-134faaedab44\") " pod="openshift-network-operator/iptables-alerter-nphrg" Jul 06 16:38:08.377666 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377567 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-etc-openvswitch\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.377666 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377601 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-etc-openvswitch\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.377666 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377612 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/84f4b2fb-a67b-4443-aba6-a54f0f17c751-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wkb7v\" (UID: \"84f4b2fb-a67b-4443-aba6-a54f0f17c751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.377666 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377639 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/66c85ad6-1024-423c-b7bc-2c17115326c1-agent-certs\") pod \"konnectivity-agent-xqf97\" (UID: \"66c85ad6-1024-423c-b7bc-2c17115326c1\") " pod="kube-system/konnectivity-agent-xqf97" Jul 06 16:38:08.377666 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377664 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-etc-sysconfig\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.378143 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377670 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/84f4b2fb-a67b-4443-aba6-a54f0f17c751-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wkb7v\" (UID: \"84f4b2fb-a67b-4443-aba6-a54f0f17c751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.378143 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377694 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/478a85da-6e22-4a45-8ab8-bd99215ec846-ovnkube-script-lib\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.378143 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377720 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/84f4b2fb-a67b-4443-aba6-a54f0f17c751-socket-dir\") pod \"aws-ebs-csi-driver-node-wkb7v\" (UID: \"84f4b2fb-a67b-4443-aba6-a54f0f17c751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.378143 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377734 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-etc-sysconfig\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.378143 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377740 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/84f4b2fb-a67b-4443-aba6-a54f0f17c751-device-dir\") pod \"aws-ebs-csi-driver-node-wkb7v\" (UID: \"84f4b2fb-a67b-4443-aba6-a54f0f17c751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.378143 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377784 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/84f4b2fb-a67b-4443-aba6-a54f0f17c751-device-dir\") pod \"aws-ebs-csi-driver-node-wkb7v\" (UID: \"84f4b2fb-a67b-4443-aba6-a54f0f17c751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.378143 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377803 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-etc-systemd\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.378143 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377924 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-etc-systemd\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.378143 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377834 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.378143 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377989 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/84f4b2fb-a67b-4443-aba6-a54f0f17c751-socket-dir\") pod \"aws-ebs-csi-driver-node-wkb7v\" (UID: \"84f4b2fb-a67b-4443-aba6-a54f0f17c751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.378143 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.377995 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/66c85ad6-1024-423c-b7bc-2c17115326c1-konnectivity-ca\") pod \"konnectivity-agent-xqf97\" (UID: \"66c85ad6-1024-423c-b7bc-2c17115326c1\") " pod="kube-system/konnectivity-agent-xqf97" Jul 06 16:38:08.378143 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378059 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-run-openvswitch\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.378143 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378067 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.378143 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378107 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-run-openvswitch\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.378143 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378129 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-run-ovn\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.378832 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378166 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/84f4b2fb-a67b-4443-aba6-a54f0f17c751-registration-dir\") pod \"aws-ebs-csi-driver-node-wkb7v\" (UID: \"84f4b2fb-a67b-4443-aba6-a54f0f17c751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.378832 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378205 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-etc-modprobe-d\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.378832 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378215 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-run-ovn\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.378832 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378262 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-run-systemd\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.378832 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378292 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/478a85da-6e22-4a45-8ab8-bd99215ec846-ovnkube-config\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.378832 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378345 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-var-lib-openvswitch\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.378832 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378353 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-etc-modprobe-d\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.378832 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378374 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-etc-sysctl-d\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.378832 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378412 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/84f4b2fb-a67b-4443-aba6-a54f0f17c751-registration-dir\") pod \"aws-ebs-csi-driver-node-wkb7v\" (UID: \"84f4b2fb-a67b-4443-aba6-a54f0f17c751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.378832 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378417 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9bd9\" (UniqueName: \"kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9\") pod \"network-check-target-xzfns\" (UID: \"52ebf132-7501-49f4-8615-28020de19a6f\") " pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:08.378832 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378442 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/478a85da-6e22-4a45-8ab8-bd99215ec846-ovnkube-script-lib\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.378832 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378448 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/84f4b2fb-a67b-4443-aba6-a54f0f17c751-sys-fs\") pod \"aws-ebs-csi-driver-node-wkb7v\" (UID: \"84f4b2fb-a67b-4443-aba6-a54f0f17c751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.378832 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378474 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-etc-sysctl-conf\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.378832 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378500 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-run-systemd\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.378832 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378518 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66cb350d-1b5e-4f99-8cee-0a68c4600341-tmp\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.378832 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378545 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ca56b80c-c9df-4f01-b5b2-134faaedab44-iptables-alerter-script\") pod \"iptables-alerter-nphrg\" (UID: \"ca56b80c-c9df-4f01-b5b2-134faaedab44\") " pod="openshift-network-operator/iptables-alerter-nphrg" Jul 06 16:38:08.378832 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378569 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-log-socket\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.379696 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378599 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-var-lib-kubelet\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.379696 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378619 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/66c85ad6-1024-423c-b7bc-2c17115326c1-konnectivity-ca\") pod \"konnectivity-agent-xqf97\" (UID: \"66c85ad6-1024-423c-b7bc-2c17115326c1\") " pod="kube-system/konnectivity-agent-xqf97" Jul 06 16:38:08.379696 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378629 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-host-cni-bin\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.379696 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378654 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkkpb\" (UniqueName: \"kubernetes.io/projected/84f4b2fb-a67b-4443-aba6-a54f0f17c751-kube-api-access-nkkpb\") pod \"aws-ebs-csi-driver-node-wkb7v\" (UID: \"84f4b2fb-a67b-4443-aba6-a54f0f17c751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.379696 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378680 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-lib-modules\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.379696 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378694 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-var-lib-openvswitch\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.379696 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378712 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-host-kubelet\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.379696 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378737 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-host-run-ovn-kubernetes\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.379696 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378755 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-host-cni-bin\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.379696 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378767 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czwvb\" (UniqueName: \"kubernetes.io/projected/478a85da-6e22-4a45-8ab8-bd99215ec846-kube-api-access-czwvb\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.379696 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378794 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0f5cbd3c-3a54-4441-bea2-2643a5912979-dbus\") pod \"global-pull-secret-syncer-2m87r\" (UID: \"0f5cbd3c-3a54-4441-bea2-2643a5912979\") " pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:08.379696 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378819 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-log-socket\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.379696 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378822 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-host-run-netns\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.379696 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378822 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-etc-sysctl-d\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.379696 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378870 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-host-run-ovn-kubernetes\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.379696 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378884 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-var-lib-kubelet\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.379696 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378889 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-lib-modules\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.380653 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378909 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-host-cni-netd\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.380653 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378939 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/478a85da-6e22-4a45-8ab8-bd99215ec846-env-overrides\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.380653 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378962 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0f5cbd3c-3a54-4441-bea2-2643a5912979-dbus\") pod \"global-pull-secret-syncer-2m87r\" (UID: \"0f5cbd3c-3a54-4441-bea2-2643a5912979\") " pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:08.380653 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.378968 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret\") pod \"global-pull-secret-syncer-2m87r\" (UID: \"0f5cbd3c-3a54-4441-bea2-2643a5912979\") " pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:08.380653 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379028 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-host-run-netns\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.380653 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:08.379055 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 16:38:08.380653 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379077 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/84f4b2fb-a67b-4443-aba6-a54f0f17c751-sys-fs\") pod \"aws-ebs-csi-driver-node-wkb7v\" (UID: \"84f4b2fb-a67b-4443-aba6-a54f0f17c751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.380653 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379099 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-etc-sysctl-conf\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.380653 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:08.379112 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret podName:0f5cbd3c-3a54-4441-bea2-2643a5912979 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:08.879092082 +0000 UTC m=+3.056951188 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret") pod "global-pull-secret-syncer-2m87r" (UID: "0f5cbd3c-3a54-4441-bea2-2643a5912979") : object "kube-system"/"original-pull-secret" not registered Jul 06 16:38:08.380653 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379120 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-host-kubelet\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.380653 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379123 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/478a85da-6e22-4a45-8ab8-bd99215ec846-ovnkube-config\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.380653 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379137 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-etc-kubernetes\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.380653 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379176 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-host-slash\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.380653 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379192 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-host-cni-netd\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.380653 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379212 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mx2tp\" (UniqueName: \"kubernetes.io/projected/ca56b80c-c9df-4f01-b5b2-134faaedab44-kube-api-access-mx2tp\") pod \"iptables-alerter-nphrg\" (UID: \"ca56b80c-c9df-4f01-b5b2-134faaedab44\") " pod="openshift-network-operator/iptables-alerter-nphrg" Jul 06 16:38:08.380653 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379242 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-systemd-units\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.380653 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379247 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ca56b80c-c9df-4f01-b5b2-134faaedab44-iptables-alerter-script\") pod \"iptables-alerter-nphrg\" (UID: \"ca56b80c-c9df-4f01-b5b2-134faaedab44\") " pod="openshift-network-operator/iptables-alerter-nphrg" Jul 06 16:38:08.381591 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379251 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-etc-kubernetes\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.381591 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379271 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/478a85da-6e22-4a45-8ab8-bd99215ec846-ovn-node-metrics-cert\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.381591 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379299 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0f5cbd3c-3a54-4441-bea2-2643a5912979-kubelet-config\") pod \"global-pull-secret-syncer-2m87r\" (UID: \"0f5cbd3c-3a54-4441-bea2-2643a5912979\") " pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:08.381591 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379308 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-host-slash\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.381591 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379324 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-run\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.381591 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379350 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-sys\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.381591 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379374 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/66cb350d-1b5e-4f99-8cee-0a68c4600341-etc-tuned\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.381591 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379402 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/84f4b2fb-a67b-4443-aba6-a54f0f17c751-etc-selinux\") pod \"aws-ebs-csi-driver-node-wkb7v\" (UID: \"84f4b2fb-a67b-4443-aba6-a54f0f17c751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.381591 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379436 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-run\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.381591 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379499 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/66cb350d-1b5e-4f99-8cee-0a68c4600341-sys\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.381591 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379509 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/478a85da-6e22-4a45-8ab8-bd99215ec846-env-overrides\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.381591 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379510 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/478a85da-6e22-4a45-8ab8-bd99215ec846-systemd-units\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.381591 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379567 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0f5cbd3c-3a54-4441-bea2-2643a5912979-kubelet-config\") pod \"global-pull-secret-syncer-2m87r\" (UID: \"0f5cbd3c-3a54-4441-bea2-2643a5912979\") " pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:08.381591 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.379580 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/84f4b2fb-a67b-4443-aba6-a54f0f17c751-etc-selinux\") pod \"aws-ebs-csi-driver-node-wkb7v\" (UID: \"84f4b2fb-a67b-4443-aba6-a54f0f17c751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.382188 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.381887 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66cb350d-1b5e-4f99-8cee-0a68c4600341-tmp\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.382188 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.381946 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/478a85da-6e22-4a45-8ab8-bd99215ec846-ovn-node-metrics-cert\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.382188 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.381968 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/66cb350d-1b5e-4f99-8cee-0a68c4600341-etc-tuned\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.382188 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.382146 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/66c85ad6-1024-423c-b7bc-2c17115326c1-agent-certs\") pod \"konnectivity-agent-xqf97\" (UID: \"66c85ad6-1024-423c-b7bc-2c17115326c1\") " pod="kube-system/konnectivity-agent-xqf97" Jul 06 16:38:08.386780 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.386755 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7ncp\" (UniqueName: \"kubernetes.io/projected/66cb350d-1b5e-4f99-8cee-0a68c4600341-kube-api-access-c7ncp\") pod \"tuned-mt8jg\" (UID: \"66cb350d-1b5e-4f99-8cee-0a68c4600341\") " pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.387669 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:08.387646 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 16:38:08.387669 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:08.387673 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 16:38:08.387808 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:08.387686 2580 projected.go:196] Error preparing data for projected volume kube-api-access-p9bd9 for pod openshift-network-diagnostics/network-check-target-xzfns: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:38:08.387808 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:08.387774 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9 podName:52ebf132-7501-49f4-8615-28020de19a6f nodeName:}" failed. No retries permitted until 2026-07-06 16:38:08.887753103 +0000 UTC m=+3.065612213 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-p9bd9" (UniqueName: "kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9") pod "network-check-target-xzfns" (UID: "52ebf132-7501-49f4-8615-28020de19a6f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:38:08.388815 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.388781 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mx2tp\" (UniqueName: \"kubernetes.io/projected/ca56b80c-c9df-4f01-b5b2-134faaedab44-kube-api-access-mx2tp\") pod \"iptables-alerter-nphrg\" (UID: \"ca56b80c-c9df-4f01-b5b2-134faaedab44\") " pod="openshift-network-operator/iptables-alerter-nphrg" Jul 06 16:38:08.390050 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.389993 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czwvb\" (UniqueName: \"kubernetes.io/projected/478a85da-6e22-4a45-8ab8-bd99215ec846-kube-api-access-czwvb\") pod \"ovnkube-node-jrb4n\" (UID: \"478a85da-6e22-4a45-8ab8-bd99215ec846\") " pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.391087 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.391069 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkkpb\" (UniqueName: \"kubernetes.io/projected/84f4b2fb-a67b-4443-aba6-a54f0f17c751-kube-api-access-nkkpb\") pod \"aws-ebs-csi-driver-node-wkb7v\" (UID: \"84f4b2fb-a67b-4443-aba6-a54f0f17c751\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.460309 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.460279 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-44nqc" Jul 06 16:38:08.466453 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.466431 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-q4flx" Jul 06 16:38:08.472628 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:08.472587 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ff8a8ec_3f37_4fc2_a436_8b1e9f1a48fb.slice/crio-063b036072714975e6fa76c3343809ff9519580695d1e55c682826a976d9a0e8 WatchSource:0}: Error finding container 063b036072714975e6fa76c3343809ff9519580695d1e55c682826a976d9a0e8: Status 404 returned error can't find the container with id 063b036072714975e6fa76c3343809ff9519580695d1e55c682826a976d9a0e8 Jul 06 16:38:08.474323 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.474117 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zw578" Jul 06 16:38:08.478690 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:08.478661 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf835c0c_f2ea_416f_a27f_619d7eeaf9c6.slice/crio-b182ac67ef0ab6b1bb1ecdff369745ead3c96edede3d90533fa3a937886a33dc WatchSource:0}: Error finding container b182ac67ef0ab6b1bb1ecdff369745ead3c96edede3d90533fa3a937886a33dc: Status 404 returned error can't find the container with id b182ac67ef0ab6b1bb1ecdff369745ead3c96edede3d90533fa3a937886a33dc Jul 06 16:38:08.480031 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.479870 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-k6c8s" Jul 06 16:38:08.485665 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.485646 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nphrg" Jul 06 16:38:08.486165 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:08.486140 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod206b0067_d25e_40fd_ab2d_5a834df0c969.slice/crio-f63c9e6db68fef48512992f73d34382b9edb27b70b66a97eddc0482b4097e652 WatchSource:0}: Error finding container f63c9e6db68fef48512992f73d34382b9edb27b70b66a97eddc0482b4097e652: Status 404 returned error can't find the container with id f63c9e6db68fef48512992f73d34382b9edb27b70b66a97eddc0482b4097e652 Jul 06 16:38:08.493103 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.491651 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:08.498482 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.498466 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" Jul 06 16:38:08.503135 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.503120 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xqf97" Jul 06 16:38:08.503396 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:08.503375 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod478a85da_6e22_4a45_8ab8_bd99215ec846.slice/crio-7579c9d87011ff3b1d840724576796b22c52443d21aa91e20f4082e64132eb85 WatchSource:0}: Error finding container 7579c9d87011ff3b1d840724576796b22c52443d21aa91e20f4082e64132eb85: Status 404 returned error can't find the container with id 7579c9d87011ff3b1d840724576796b22c52443d21aa91e20f4082e64132eb85 Jul 06 16:38:08.509219 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.509004 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" Jul 06 16:38:08.510681 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:08.510654 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84f4b2fb_a67b_4443_aba6_a54f0f17c751.slice/crio-e712cfd1ce869c0f40ee21f768aa93150aa9ff737ea83658e0ac0605d124ede7 WatchSource:0}: Error finding container e712cfd1ce869c0f40ee21f768aa93150aa9ff737ea83658e0ac0605d124ede7: Status 404 returned error can't find the container with id e712cfd1ce869c0f40ee21f768aa93150aa9ff737ea83658e0ac0605d124ede7 Jul 06 16:38:08.517661 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:08.517634 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66c85ad6_1024_423c_b7bc_2c17115326c1.slice/crio-8c3aedefd94c0c7f28a5748040731a3bd54cd7d7db5ac8e239c60778b2da4f22 WatchSource:0}: Error finding container 8c3aedefd94c0c7f28a5748040731a3bd54cd7d7db5ac8e239c60778b2da4f22: Status 404 returned error can't find the container with id 8c3aedefd94c0c7f28a5748040731a3bd54cd7d7db5ac8e239c60778b2da4f22 Jul 06 16:38:08.525011 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:08.524971 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66cb350d_1b5e_4f99_8cee_0a68c4600341.slice/crio-8cb4e2c8a1920f35dd5a801b99d62d90f5b7a2a9f3f71e724383451cdd649941 WatchSource:0}: Error finding container 8cb4e2c8a1920f35dd5a801b99d62d90f5b7a2a9f3f71e724383451cdd649941: Status 404 returned error can't find the container with id 8cb4e2c8a1920f35dd5a801b99d62d90f5b7a2a9f3f71e724383451cdd649941 Jul 06 16:38:08.549546 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.549513 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 16:38:08.783937 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.783847 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs\") pod \"network-metrics-daemon-5m692\" (UID: \"3f867a4a-dbf4-46a7-a79b-c30d916fa0f8\") " pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:08.784146 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:08.784012 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:38:08.784146 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:08.784082 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs podName:3f867a4a-dbf4-46a7-a79b-c30d916fa0f8 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:09.784064849 +0000 UTC m=+3.961923942 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs") pod "network-metrics-daemon-5m692" (UID: "3f867a4a-dbf4-46a7-a79b-c30d916fa0f8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:38:08.884705 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.884672 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret\") pod \"global-pull-secret-syncer-2m87r\" (UID: \"0f5cbd3c-3a54-4441-bea2-2643a5912979\") " pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:08.884901 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:08.884856 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 16:38:08.884972 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:08.884930 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret podName:0f5cbd3c-3a54-4441-bea2-2643a5912979 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:09.884909799 +0000 UTC m=+4.062768907 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret") pod "global-pull-secret-syncer-2m87r" (UID: "0f5cbd3c-3a54-4441-bea2-2643a5912979") : object "kube-system"/"original-pull-secret" not registered Jul 06 16:38:08.985345 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:08.985305 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9bd9\" (UniqueName: \"kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9\") pod \"network-check-target-xzfns\" (UID: \"52ebf132-7501-49f4-8615-28020de19a6f\") " pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:08.985560 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:08.985490 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 16:38:08.985560 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:08.985528 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 16:38:08.985560 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:08.985542 2580 projected.go:196] Error preparing data for projected volume kube-api-access-p9bd9 for pod openshift-network-diagnostics/network-check-target-xzfns: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:38:08.985743 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:08.985616 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9 podName:52ebf132-7501-49f4-8615-28020de19a6f nodeName:}" failed. No retries permitted until 2026-07-06 16:38:09.985592263 +0000 UTC m=+4.163451357 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-p9bd9" (UniqueName: "kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9") pod "network-check-target-xzfns" (UID: "52ebf132-7501-49f4-8615-28020de19a6f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:38:09.230867 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:09.230759 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 16:33:08 +0000 UTC" deadline="2028-01-20 12:57:49.453452281 +0000 UTC" Jul 06 16:38:09.230867 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:09.230817 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13508h19m40.222640933s" Jul 06 16:38:09.259294 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:09.259255 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:09.259479 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:09.259426 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2m87r" podUID="0f5cbd3c-3a54-4441-bea2-2643a5912979" Jul 06 16:38:09.268992 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:09.268960 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zw578" event={"ID":"206b0067-d25e-40fd-ab2d-5a834df0c969","Type":"ContainerStarted","Data":"f63c9e6db68fef48512992f73d34382b9edb27b70b66a97eddc0482b4097e652"} Jul 06 16:38:09.270278 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:09.270247 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" event={"ID":"66cb350d-1b5e-4f99-8cee-0a68c4600341","Type":"ContainerStarted","Data":"8cb4e2c8a1920f35dd5a801b99d62d90f5b7a2a9f3f71e724383451cdd649941"} Jul 06 16:38:09.271435 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:09.271411 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xqf97" event={"ID":"66c85ad6-1024-423c-b7bc-2c17115326c1","Type":"ContainerStarted","Data":"8c3aedefd94c0c7f28a5748040731a3bd54cd7d7db5ac8e239c60778b2da4f22"} Jul 06 16:38:09.272655 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:09.272628 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" event={"ID":"478a85da-6e22-4a45-8ab8-bd99215ec846","Type":"ContainerStarted","Data":"7579c9d87011ff3b1d840724576796b22c52443d21aa91e20f4082e64132eb85"} Jul 06 16:38:09.273741 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:09.273717 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nphrg" event={"ID":"ca56b80c-c9df-4f01-b5b2-134faaedab44","Type":"ContainerStarted","Data":"98f35f091659a2eaa892cd76cd26d72c40a77dbdaa314b2934b5427a0bc2be03"} Jul 06 16:38:09.274938 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:09.274912 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k6c8s" event={"ID":"e07c98e1-5907-417d-91c3-ffd8a87fab85","Type":"ContainerStarted","Data":"e7f57864dc62cafa1c56e8817c67b794f6d2b18f919bbfad339a6b77d465211b"} Jul 06 16:38:09.276086 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:09.276065 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q4flx" event={"ID":"cf835c0c-f2ea-416f-a27f-619d7eeaf9c6","Type":"ContainerStarted","Data":"b182ac67ef0ab6b1bb1ecdff369745ead3c96edede3d90533fa3a937886a33dc"} Jul 06 16:38:09.277172 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:09.277148 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44nqc" event={"ID":"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb","Type":"ContainerStarted","Data":"063b036072714975e6fa76c3343809ff9519580695d1e55c682826a976d9a0e8"} Jul 06 16:38:09.278421 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:09.278398 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" event={"ID":"84f4b2fb-a67b-4443-aba6-a54f0f17c751","Type":"ContainerStarted","Data":"e712cfd1ce869c0f40ee21f768aa93150aa9ff737ea83658e0ac0605d124ede7"} Jul 06 16:38:09.793555 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:09.793526 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs\") pod \"network-metrics-daemon-5m692\" (UID: \"3f867a4a-dbf4-46a7-a79b-c30d916fa0f8\") " pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:09.794044 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:09.793661 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:38:09.794044 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:09.793727 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs podName:3f867a4a-dbf4-46a7-a79b-c30d916fa0f8 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:11.793708294 +0000 UTC m=+5.971567393 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs") pod "network-metrics-daemon-5m692" (UID: "3f867a4a-dbf4-46a7-a79b-c30d916fa0f8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:38:09.894289 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:09.894258 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret\") pod \"global-pull-secret-syncer-2m87r\" (UID: \"0f5cbd3c-3a54-4441-bea2-2643a5912979\") " pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:09.894473 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:09.894451 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 16:38:09.894597 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:09.894547 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret podName:0f5cbd3c-3a54-4441-bea2-2643a5912979 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:11.894525533 +0000 UTC m=+6.072384631 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret") pod "global-pull-secret-syncer-2m87r" (UID: "0f5cbd3c-3a54-4441-bea2-2643a5912979") : object "kube-system"/"original-pull-secret" not registered Jul 06 16:38:09.996346 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:09.995390 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9bd9\" (UniqueName: \"kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9\") pod \"network-check-target-xzfns\" (UID: \"52ebf132-7501-49f4-8615-28020de19a6f\") " pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:09.996346 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:09.995883 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 16:38:09.996346 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:09.995914 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 16:38:09.996346 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:09.995929 2580 projected.go:196] Error preparing data for projected volume kube-api-access-p9bd9 for pod openshift-network-diagnostics/network-check-target-xzfns: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:38:09.996346 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:09.995988 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9 podName:52ebf132-7501-49f4-8615-28020de19a6f nodeName:}" failed. No retries permitted until 2026-07-06 16:38:11.995969492 +0000 UTC m=+6.173828587 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-p9bd9" (UniqueName: "kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9") pod "network-check-target-xzfns" (UID: "52ebf132-7501-49f4-8615-28020de19a6f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:38:10.231533 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:10.231391 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 16:33:08 +0000 UTC" deadline="2028-02-08 20:51:40.197403792 +0000 UTC" Jul 06 16:38:10.231533 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:10.231434 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13972h13m29.965974967s" Jul 06 16:38:10.261151 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:10.258866 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:10.261151 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:10.259031 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m692" podUID="3f867a4a-dbf4-46a7-a79b-c30d916fa0f8" Jul 06 16:38:10.261151 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:10.259565 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:10.261151 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:10.259672 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzfns" podUID="52ebf132-7501-49f4-8615-28020de19a6f" Jul 06 16:38:10.297571 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:10.296523 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal" event={"ID":"b5e29f9f4d849ce3821c6ee834e1493a","Type":"ContainerStarted","Data":"acf0c15a07012123f4fb581d8f04823dadf74cecde18b7432a0cc0a419cfe3bc"} Jul 06 16:38:11.259071 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:11.259041 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:11.259599 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:11.259205 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2m87r" podUID="0f5cbd3c-3a54-4441-bea2-2643a5912979" Jul 06 16:38:11.812439 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:11.812397 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs\") pod \"network-metrics-daemon-5m692\" (UID: \"3f867a4a-dbf4-46a7-a79b-c30d916fa0f8\") " pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:11.812643 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:11.812623 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:38:11.812711 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:11.812686 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs podName:3f867a4a-dbf4-46a7-a79b-c30d916fa0f8 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:15.812668147 +0000 UTC m=+9.990527259 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs") pod "network-metrics-daemon-5m692" (UID: "3f867a4a-dbf4-46a7-a79b-c30d916fa0f8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:38:11.913926 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:11.913861 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret\") pod \"global-pull-secret-syncer-2m87r\" (UID: \"0f5cbd3c-3a54-4441-bea2-2643a5912979\") " pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:11.914112 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:11.914000 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 16:38:11.914112 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:11.914062 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret podName:0f5cbd3c-3a54-4441-bea2-2643a5912979 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:15.914043976 +0000 UTC m=+10.091903072 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret") pod "global-pull-secret-syncer-2m87r" (UID: "0f5cbd3c-3a54-4441-bea2-2643a5912979") : object "kube-system"/"original-pull-secret" not registered Jul 06 16:38:12.014839 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:12.014800 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9bd9\" (UniqueName: \"kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9\") pod \"network-check-target-xzfns\" (UID: \"52ebf132-7501-49f4-8615-28020de19a6f\") " pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:12.014997 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:12.014987 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 16:38:12.015055 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:12.015004 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 16:38:12.015055 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:12.015016 2580 projected.go:196] Error preparing data for projected volume kube-api-access-p9bd9 for pod openshift-network-diagnostics/network-check-target-xzfns: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:38:12.015166 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:12.015073 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9 podName:52ebf132-7501-49f4-8615-28020de19a6f nodeName:}" failed. No retries permitted until 2026-07-06 16:38:16.015055848 +0000 UTC m=+10.192914952 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-p9bd9" (UniqueName: "kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9") pod "network-check-target-xzfns" (UID: "52ebf132-7501-49f4-8615-28020de19a6f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:38:12.259225 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:12.259150 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:12.259671 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:12.259285 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzfns" podUID="52ebf132-7501-49f4-8615-28020de19a6f" Jul 06 16:38:12.259671 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:12.259513 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:12.259671 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:12.259659 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m692" podUID="3f867a4a-dbf4-46a7-a79b-c30d916fa0f8" Jul 06 16:38:13.259128 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:13.259095 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:13.259298 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:13.259229 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2m87r" podUID="0f5cbd3c-3a54-4441-bea2-2643a5912979" Jul 06 16:38:14.260066 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:14.259413 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:14.260066 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:14.259449 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:14.260066 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:14.259572 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m692" podUID="3f867a4a-dbf4-46a7-a79b-c30d916fa0f8" Jul 06 16:38:14.260066 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:14.259696 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzfns" podUID="52ebf132-7501-49f4-8615-28020de19a6f" Jul 06 16:38:15.259695 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:15.259185 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:15.259695 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:15.259319 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2m87r" podUID="0f5cbd3c-3a54-4441-bea2-2643a5912979" Jul 06 16:38:15.841111 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:15.841073 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs\") pod \"network-metrics-daemon-5m692\" (UID: \"3f867a4a-dbf4-46a7-a79b-c30d916fa0f8\") " pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:15.841584 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:15.841279 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:38:15.841584 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:15.841360 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs podName:3f867a4a-dbf4-46a7-a79b-c30d916fa0f8 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:23.841338512 +0000 UTC m=+18.019197607 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs") pod "network-metrics-daemon-5m692" (UID: "3f867a4a-dbf4-46a7-a79b-c30d916fa0f8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:38:15.941869 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:15.941831 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret\") pod \"global-pull-secret-syncer-2m87r\" (UID: \"0f5cbd3c-3a54-4441-bea2-2643a5912979\") " pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:15.942052 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:15.941970 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 16:38:15.942052 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:15.942028 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret podName:0f5cbd3c-3a54-4441-bea2-2643a5912979 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:23.94201479 +0000 UTC m=+18.119873880 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret") pod "global-pull-secret-syncer-2m87r" (UID: "0f5cbd3c-3a54-4441-bea2-2643a5912979") : object "kube-system"/"original-pull-secret" not registered Jul 06 16:38:16.042823 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:16.042788 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9bd9\" (UniqueName: \"kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9\") pod \"network-check-target-xzfns\" (UID: \"52ebf132-7501-49f4-8615-28020de19a6f\") " pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:16.042983 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:16.042969 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 16:38:16.043053 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:16.042995 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 16:38:16.043053 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:16.043008 2580 projected.go:196] Error preparing data for projected volume kube-api-access-p9bd9 for pod openshift-network-diagnostics/network-check-target-xzfns: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:38:16.043127 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:16.043071 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9 podName:52ebf132-7501-49f4-8615-28020de19a6f nodeName:}" failed. No retries permitted until 2026-07-06 16:38:24.043052015 +0000 UTC m=+18.220911124 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-p9bd9" (UniqueName: "kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9") pod "network-check-target-xzfns" (UID: "52ebf132-7501-49f4-8615-28020de19a6f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:38:16.259708 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:16.259171 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:16.259708 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:16.259302 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m692" podUID="3f867a4a-dbf4-46a7-a79b-c30d916fa0f8" Jul 06 16:38:16.259708 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:16.259364 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:16.259708 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:16.259485 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzfns" podUID="52ebf132-7501-49f4-8615-28020de19a6f" Jul 06 16:38:17.258599 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:17.258567 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:17.259056 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:17.258706 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2m87r" podUID="0f5cbd3c-3a54-4441-bea2-2643a5912979" Jul 06 16:38:17.325095 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:17.325004 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xqf97" event={"ID":"66c85ad6-1024-423c-b7bc-2c17115326c1","Type":"ContainerStarted","Data":"3713a9f3a266832363c1221028f96803347d502cf0f4a94f018bc7b2926964e6"} Jul 06 16:38:17.327215 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:17.326861 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-q4flx" event={"ID":"cf835c0c-f2ea-416f-a27f-619d7eeaf9c6","Type":"ContainerStarted","Data":"65df4c0fc79ce57346d2f84dda690a10a172112cf9ce5bd82289af0c53e7c8a6"} Jul 06 16:38:17.329550 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:17.329106 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44nqc" event={"ID":"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb","Type":"ContainerStarted","Data":"cc697800f4e27edcdd7ede66e073166a6915fb7e525fbfcadafd9e4d2a0fee24"} Jul 06 16:38:17.331893 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:17.331297 2580 generic.go:356] "Generic (PLEG): container finished" podID="e07cd2e8cb8f8817f3a6b417cfd6e9f6" containerID="61c6dfae38841f331c7dd081ff73d553509b764478eb4fcf6fe3180534566927" exitCode=0 Jul 06 16:38:17.331893 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:17.331413 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal" event={"ID":"e07cd2e8cb8f8817f3a6b417cfd6e9f6","Type":"ContainerDied","Data":"61c6dfae38841f331c7dd081ff73d553509b764478eb4fcf6fe3180534566927"} Jul 06 16:38:17.333838 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:17.333813 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" event={"ID":"84f4b2fb-a67b-4443-aba6-a54f0f17c751","Type":"ContainerStarted","Data":"646c77571415c905f5d57d167ba88d5a0353ec4d3b75c761f86ae36170423f4c"} Jul 06 16:38:17.336334 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:17.336310 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zw578" event={"ID":"206b0067-d25e-40fd-ab2d-5a834df0c969","Type":"ContainerStarted","Data":"f7903e1b6a5eaef76da85f914e44bc269d9c34a7438e6c4d773d4f34568c19af"} Jul 06 16:38:17.347549 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:17.347495 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-40.ec2.internal" podStartSLOduration=10.347463664 podStartE2EDuration="10.347463664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:38:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:38:10.33004975 +0000 UTC m=+4.507908866" watchObservedRunningTime="2026-07-06 16:38:17.347463664 +0000 UTC m=+11.525322780" Jul 06 16:38:17.370725 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:17.370576 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xqf97" podStartSLOduration=2.534169241 podStartE2EDuration="10.370563382s" totalImagesPullingTime="7.836394141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:38:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:38:17.347459078 +0000 UTC m=+11.525318191" watchObservedRunningTime="2026-07-06 16:38:17.370563382 +0000 UTC m=+11.548422494" Jul 06 16:38:17.391097 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:17.391040 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-q4flx" podStartSLOduration=3.505454656 podStartE2EDuration="11.3910233s" totalImagesPullingTime="7.885568644s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:38:17.39089302 +0000 UTC m=+11.568752134" watchObservedRunningTime="2026-07-06 16:38:17.3910233 +0000 UTC m=+11.568882413" Jul 06 16:38:18.259195 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:18.259104 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:18.259671 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:18.259243 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzfns" podUID="52ebf132-7501-49f4-8615-28020de19a6f" Jul 06 16:38:18.259671 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:18.259319 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:18.259671 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:18.259436 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m692" podUID="3f867a4a-dbf4-46a7-a79b-c30d916fa0f8" Jul 06 16:38:18.341050 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:18.340585 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nphrg" event={"ID":"ca56b80c-c9df-4f01-b5b2-134faaedab44","Type":"ContainerStarted","Data":"9f51ef81a194ff206493eb87ec4f019280f61069cd2d32bc63a48854138628d0"} Jul 06 16:38:18.362132 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:18.361525 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zw578" podStartSLOduration=4.494671389 podStartE2EDuration="12.36148834s" totalImagesPullingTime="7.866816951s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:38:17.444760818 +0000 UTC m=+11.622619933" watchObservedRunningTime="2026-07-06 16:38:18.36148834 +0000 UTC m=+12.539347452" Jul 06 16:38:18.362132 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:18.361779 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nphrg" podStartSLOduration=4.504756696 podStartE2EDuration="12.361772081s" totalImagesPullingTime="7.857015385s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:38:18.361425029 +0000 UTC m=+12.539284145" watchObservedRunningTime="2026-07-06 16:38:18.361772081 +0000 UTC m=+12.539631194" Jul 06 16:38:19.258799 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:19.258612 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:19.258927 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:19.258884 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2m87r" podUID="0f5cbd3c-3a54-4441-bea2-2643a5912979" Jul 06 16:38:19.345208 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:19.345181 2580 generic.go:356] "Generic (PLEG): container finished" podID="6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb" containerID="cc697800f4e27edcdd7ede66e073166a6915fb7e525fbfcadafd9e4d2a0fee24" exitCode=0 Jul 06 16:38:19.345588 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:19.345273 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44nqc" event={"ID":"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb","Type":"ContainerDied","Data":"cc697800f4e27edcdd7ede66e073166a6915fb7e525fbfcadafd9e4d2a0fee24"} Jul 06 16:38:19.347951 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:19.347925 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal" event={"ID":"e07cd2e8cb8f8817f3a6b417cfd6e9f6","Type":"ContainerStarted","Data":"52c245f676cae07d02df8e07c8794d282092dc53dccba6a092616806dea420a9"} Jul 06 16:38:20.250861 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:20.250830 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 16:38:20.259016 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:20.258995 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:20.259151 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:20.259129 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m692" podUID="3f867a4a-dbf4-46a7-a79b-c30d916fa0f8" Jul 06 16:38:20.259213 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:20.259167 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:20.259270 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:20.259239 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzfns" podUID="52ebf132-7501-49f4-8615-28020de19a6f" Jul 06 16:38:20.353321 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:20.353289 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" event={"ID":"84f4b2fb-a67b-4443-aba6-a54f0f17c751","Type":"ContainerStarted","Data":"2fa77de4e00838db5e41feaef5ffbc03d8bc6ab57478b5eeb135a1c1579b49e7"} Jul 06 16:38:20.355066 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:20.355028 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" event={"ID":"66cb350d-1b5e-4f99-8cee-0a68c4600341","Type":"ContainerStarted","Data":"fd85630120370ee913d01a6c023803bba903558ab0eb534efef8eacba022cc5a"} Jul 06 16:38:20.385020 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:20.384973 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-40.ec2.internal" podStartSLOduration=13.384956715 podStartE2EDuration="13.384956715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:38:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:38:19.402543004 +0000 UTC m=+13.580402114" watchObservedRunningTime="2026-07-06 16:38:20.384956715 +0000 UTC m=+14.562815830" Jul 06 16:38:20.385429 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:20.385397 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mt8jg" podStartSLOduration=2.606629451 podStartE2EDuration="13.385385935s" totalImagesPullingTime="10.778756484s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:38:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:38:20.384114317 +0000 UTC m=+14.561973430" watchObservedRunningTime="2026-07-06 16:38:20.385385935 +0000 UTC m=+14.563245050" Jul 06 16:38:21.243156 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:21.243061 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T16:38:20.250854439Z","UUID":"23e1378f-5bac-483f-a6a7-e7b5762cd566","Handler":null,"Name":"","Endpoint":""} Jul 06 16:38:21.244834 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:21.244804 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 16:38:21.244953 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:21.244850 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 16:38:21.258840 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:21.258812 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:21.258948 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:21.258919 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2m87r" podUID="0f5cbd3c-3a54-4441-bea2-2643a5912979" Jul 06 16:38:21.431103 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:21.431076 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xqf97" Jul 06 16:38:21.431783 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:21.431735 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xqf97" Jul 06 16:38:22.258961 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:22.258930 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:22.259123 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:22.258930 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:22.259123 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:22.259069 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m692" podUID="3f867a4a-dbf4-46a7-a79b-c30d916fa0f8" Jul 06 16:38:22.259123 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:22.259114 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzfns" podUID="52ebf132-7501-49f4-8615-28020de19a6f" Jul 06 16:38:23.259173 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:23.258923 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:23.259659 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:23.259265 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2m87r" podUID="0f5cbd3c-3a54-4441-bea2-2643a5912979" Jul 06 16:38:23.908891 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:23.908847 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs\") pod \"network-metrics-daemon-5m692\" (UID: \"3f867a4a-dbf4-46a7-a79b-c30d916fa0f8\") " pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:23.909074 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:23.909003 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:38:23.909145 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:23.909080 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs podName:3f867a4a-dbf4-46a7-a79b-c30d916fa0f8 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:39.909060779 +0000 UTC m=+34.086919875 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs") pod "network-metrics-daemon-5m692" (UID: "3f867a4a-dbf4-46a7-a79b-c30d916fa0f8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:38:24.010044 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:24.010003 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret\") pod \"global-pull-secret-syncer-2m87r\" (UID: \"0f5cbd3c-3a54-4441-bea2-2643a5912979\") " pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:24.010219 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:24.010120 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 16:38:24.010219 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:24.010197 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret podName:0f5cbd3c-3a54-4441-bea2-2643a5912979 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:40.010176016 +0000 UTC m=+34.188035107 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret") pod "global-pull-secret-syncer-2m87r" (UID: "0f5cbd3c-3a54-4441-bea2-2643a5912979") : object "kube-system"/"original-pull-secret" not registered Jul 06 16:38:24.110841 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:24.110806 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9bd9\" (UniqueName: \"kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9\") pod \"network-check-target-xzfns\" (UID: \"52ebf132-7501-49f4-8615-28020de19a6f\") " pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:24.111010 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:24.110940 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 16:38:24.111010 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:24.110957 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 16:38:24.111010 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:24.110969 2580 projected.go:196] Error preparing data for projected volume kube-api-access-p9bd9 for pod openshift-network-diagnostics/network-check-target-xzfns: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:38:24.111167 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:24.111030 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9 podName:52ebf132-7501-49f4-8615-28020de19a6f nodeName:}" failed. No retries permitted until 2026-07-06 16:38:40.1110127 +0000 UTC m=+34.288871806 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-p9bd9" (UniqueName: "kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9") pod "network-check-target-xzfns" (UID: "52ebf132-7501-49f4-8615-28020de19a6f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:38:24.258664 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:24.258576 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:24.258664 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:24.258592 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:24.258895 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:24.258706 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzfns" podUID="52ebf132-7501-49f4-8615-28020de19a6f" Jul 06 16:38:24.258895 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:24.258862 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m692" podUID="3f867a4a-dbf4-46a7-a79b-c30d916fa0f8" Jul 06 16:38:25.259151 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:25.259117 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:25.259630 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:25.259246 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2m87r" podUID="0f5cbd3c-3a54-4441-bea2-2643a5912979" Jul 06 16:38:26.259342 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:26.259313 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:26.259778 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:26.259344 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:26.259778 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:26.259451 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzfns" podUID="52ebf132-7501-49f4-8615-28020de19a6f" Jul 06 16:38:26.259778 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:26.259571 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m692" podUID="3f867a4a-dbf4-46a7-a79b-c30d916fa0f8" Jul 06 16:38:27.259282 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:27.259242 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:27.259458 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:27.259378 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2m87r" podUID="0f5cbd3c-3a54-4441-bea2-2643a5912979" Jul 06 16:38:28.259360 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:28.259315 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:28.259546 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:28.259450 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzfns" podUID="52ebf132-7501-49f4-8615-28020de19a6f" Jul 06 16:38:28.259546 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:28.259521 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:28.259977 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:28.259614 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m692" podUID="3f867a4a-dbf4-46a7-a79b-c30d916fa0f8" Jul 06 16:38:29.259294 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:29.259264 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:29.259485 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:29.259368 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2m87r" podUID="0f5cbd3c-3a54-4441-bea2-2643a5912979" Jul 06 16:38:29.865367 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:29.865221 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xqf97" Jul 06 16:38:29.865697 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:29.865464 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 06 16:38:29.865796 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:29.865780 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xqf97" Jul 06 16:38:30.258791 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:30.258726 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:30.258938 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:30.258726 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:30.258938 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:30.258853 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m692" podUID="3f867a4a-dbf4-46a7-a79b-c30d916fa0f8" Jul 06 16:38:30.258938 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:30.258891 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzfns" podUID="52ebf132-7501-49f4-8615-28020de19a6f" Jul 06 16:38:30.378373 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:30.378345 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 16:38:30.378765 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:30.378741 2580 generic.go:356] "Generic (PLEG): container finished" podID="478a85da-6e22-4a45-8ab8-bd99215ec846" containerID="b5878e273b61739cffe11d238508f33185a6a4c34901dc9125eeb448f15eff2a" exitCode=1 Jul 06 16:38:30.378871 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:30.378829 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" event={"ID":"478a85da-6e22-4a45-8ab8-bd99215ec846","Type":"ContainerStarted","Data":"327e2ab52a36f30ab926d97c585831e3e653870d528ca760c846bf8e8b79f2d1"} Jul 06 16:38:30.378929 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:30.378918 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" event={"ID":"478a85da-6e22-4a45-8ab8-bd99215ec846","Type":"ContainerStarted","Data":"930e9f694ff8c8ee1834b61e18d496471926c59472b8afcb826c3780f4f42310"} Jul 06 16:38:30.378968 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:30.378957 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" event={"ID":"478a85da-6e22-4a45-8ab8-bd99215ec846","Type":"ContainerDied","Data":"b5878e273b61739cffe11d238508f33185a6a4c34901dc9125eeb448f15eff2a"} Jul 06 16:38:30.379010 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:30.378989 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" event={"ID":"478a85da-6e22-4a45-8ab8-bd99215ec846","Type":"ContainerStarted","Data":"83842ec2f4cc5865504e18542872d2374e77af460e663634e668b1c4252a2aea"} Jul 06 16:38:30.380440 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:30.380412 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-k6c8s" event={"ID":"e07c98e1-5907-417d-91c3-ffd8a87fab85","Type":"ContainerStarted","Data":"0a2fd49d7af8943a36bf6c476e306a64874e25b9bff7f8b5442bbd5515085fb5"} Jul 06 16:38:30.382722 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:30.382699 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" event={"ID":"84f4b2fb-a67b-4443-aba6-a54f0f17c751","Type":"ContainerStarted","Data":"c1370c195a433000e9eb1aad312e2c7c646f527603951feb4d6cdae98c9f099f"} Jul 06 16:38:30.443454 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:30.443400 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-k6c8s" podStartSLOduration=3.540843981 podStartE2EDuration="24.44337848s" totalImagesPullingTime="20.902534499s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:38:30.4433073 +0000 UTC m=+24.621166411" watchObservedRunningTime="2026-07-06 16:38:30.44337848 +0000 UTC m=+24.621237593" Jul 06 16:38:30.478838 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:30.478785 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wkb7v" podStartSLOduration=3.051038149 podStartE2EDuration="23.478768208s" totalImagesPullingTime="20.427730059s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:38:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:38:30.478220015 +0000 UTC m=+24.656079204" watchObservedRunningTime="2026-07-06 16:38:30.478768208 +0000 UTC m=+24.656627319" Jul 06 16:38:31.258997 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:31.258960 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:31.259577 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:31.259091 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2m87r" podUID="0f5cbd3c-3a54-4441-bea2-2643a5912979" Jul 06 16:38:32.259026 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:32.258951 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:32.259502 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:32.258950 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:32.259502 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:32.259146 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m692" podUID="3f867a4a-dbf4-46a7-a79b-c30d916fa0f8" Jul 06 16:38:32.259502 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:32.259042 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzfns" podUID="52ebf132-7501-49f4-8615-28020de19a6f" Jul 06 16:38:32.388659 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:32.388636 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 16:38:32.389026 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:32.389005 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" event={"ID":"478a85da-6e22-4a45-8ab8-bd99215ec846","Type":"ContainerStarted","Data":"34a342d62f519bdc0ad8419aabddaa7c95ed54e8c6133cf6e84d548320c72b30"} Jul 06 16:38:32.389095 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:32.389080 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" event={"ID":"478a85da-6e22-4a45-8ab8-bd99215ec846","Type":"ContainerStarted","Data":"112fff794fdbb437521837c42ec49c800bb92a55b01dae3ce88e1adac4b42507"} Jul 06 16:38:32.390580 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:32.390557 2580 generic.go:356] "Generic (PLEG): container finished" podID="6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb" containerID="2d86c274a86215f44b2587e423ba96ae772bd60795d502be33046ce47e0379f4" exitCode=0 Jul 06 16:38:32.390669 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:32.390601 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44nqc" event={"ID":"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb","Type":"ContainerDied","Data":"2d86c274a86215f44b2587e423ba96ae772bd60795d502be33046ce47e0379f4"} Jul 06 16:38:33.259236 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:33.259201 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:33.259706 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:33.259303 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2m87r" podUID="0f5cbd3c-3a54-4441-bea2-2643a5912979" Jul 06 16:38:34.258450 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:34.258380 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:34.258589 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:34.258498 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzfns" podUID="52ebf132-7501-49f4-8615-28020de19a6f" Jul 06 16:38:34.258589 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:34.258529 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:34.258589 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:34.258583 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m692" podUID="3f867a4a-dbf4-46a7-a79b-c30d916fa0f8" Jul 06 16:38:34.396888 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:34.396857 2580 generic.go:356] "Generic (PLEG): container finished" podID="6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb" containerID="7e0392b5b36a3b5332f2ef8286ae4ce6f941c01f8541c0016224bef93b97e277" exitCode=0 Jul 06 16:38:34.397338 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:34.396928 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44nqc" event={"ID":"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb","Type":"ContainerDied","Data":"7e0392b5b36a3b5332f2ef8286ae4ce6f941c01f8541c0016224bef93b97e277"} Jul 06 16:38:34.399790 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:34.399771 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 16:38:34.400100 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:34.400081 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" event={"ID":"478a85da-6e22-4a45-8ab8-bd99215ec846","Type":"ContainerStarted","Data":"358ec4b902e7a2e06cd11090f0bb2637ae208742dfc05efe71cf4ed7ed13ae36"} Jul 06 16:38:35.258838 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:35.258807 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:35.258990 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:35.258912 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2m87r" podUID="0f5cbd3c-3a54-4441-bea2-2643a5912979" Jul 06 16:38:36.259859 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:36.259777 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:36.260241 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:36.259882 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m692" podUID="3f867a4a-dbf4-46a7-a79b-c30d916fa0f8" Jul 06 16:38:36.260241 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:36.259929 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:36.260241 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:36.260006 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzfns" podUID="52ebf132-7501-49f4-8615-28020de19a6f" Jul 06 16:38:36.408185 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:36.408156 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 16:38:36.408542 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:36.408509 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" event={"ID":"478a85da-6e22-4a45-8ab8-bd99215ec846","Type":"ContainerStarted","Data":"76f234b30b9d1e3877c6f9f980e02e4ec3f67ea29a56209d777112bff8e347ec"} Jul 06 16:38:36.408772 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:36.408755 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:36.408985 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:36.408967 2580 scope.go:117] "RemoveContainer" containerID="b5878e273b61739cffe11d238508f33185a6a4c34901dc9125eeb448f15eff2a" Jul 06 16:38:36.410955 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:36.410928 2580 generic.go:356] "Generic (PLEG): container finished" podID="6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb" containerID="4da339fd47641a4cc3310096a428105842a131ccd8e785199890db992dcfec6f" exitCode=0 Jul 06 16:38:36.411060 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:36.410971 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44nqc" event={"ID":"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb","Type":"ContainerDied","Data":"4da339fd47641a4cc3310096a428105842a131ccd8e785199890db992dcfec6f"} Jul 06 16:38:36.425285 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:36.425263 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:37.259027 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:37.258994 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:37.259208 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:37.259104 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2m87r" podUID="0f5cbd3c-3a54-4441-bea2-2643a5912979" Jul 06 16:38:37.422172 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:37.422090 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 16:38:37.424382 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:37.422987 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" event={"ID":"478a85da-6e22-4a45-8ab8-bd99215ec846","Type":"ContainerStarted","Data":"eca44524cfb29ec18057fae6a4b386d85dbb809064a6999e5541621ad348e21e"} Jul 06 16:38:37.424382 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:37.424232 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:37.424382 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:37.424312 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:37.447463 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:37.447431 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:38:37.535831 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:37.535723 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" podStartSLOduration=10.597992849 podStartE2EDuration="31.535704685s" totalImagesPullingTime="20.937711836s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:38:37.487547681 +0000 UTC m=+31.665406792" watchObservedRunningTime="2026-07-06 16:38:37.535704685 +0000 UTC m=+31.713563796" Jul 06 16:38:38.259008 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:38.258977 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:38.259145 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:38.259118 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m692" podUID="3f867a4a-dbf4-46a7-a79b-c30d916fa0f8" Jul 06 16:38:38.259196 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:38.259160 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:38.259270 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:38.259252 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzfns" podUID="52ebf132-7501-49f4-8615-28020de19a6f" Jul 06 16:38:38.703488 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:38.703432 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5m692"] Jul 06 16:38:38.703933 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:38.703588 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:38.703933 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:38.703701 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m692" podUID="3f867a4a-dbf4-46a7-a79b-c30d916fa0f8" Jul 06 16:38:38.708144 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:38.708120 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2m87r"] Jul 06 16:38:38.708275 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:38.708204 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:38.708343 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:38.708287 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2m87r" podUID="0f5cbd3c-3a54-4441-bea2-2643a5912979" Jul 06 16:38:38.712673 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:38.712652 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xzfns"] Jul 06 16:38:38.712774 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:38.712729 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:38.712841 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:38.712799 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzfns" podUID="52ebf132-7501-49f4-8615-28020de19a6f" Jul 06 16:38:39.929418 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:39.929382 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs\") pod \"network-metrics-daemon-5m692\" (UID: \"3f867a4a-dbf4-46a7-a79b-c30d916fa0f8\") " pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:39.929967 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:39.929539 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:38:39.929967 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:39.929602 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs podName:3f867a4a-dbf4-46a7-a79b-c30d916fa0f8 nodeName:}" failed. No retries permitted until 2026-07-06 16:39:11.929587506 +0000 UTC m=+66.107446598 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs") pod "network-metrics-daemon-5m692" (UID: "3f867a4a-dbf4-46a7-a79b-c30d916fa0f8") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 16:38:40.030289 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:40.030254 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret\") pod \"global-pull-secret-syncer-2m87r\" (UID: \"0f5cbd3c-3a54-4441-bea2-2643a5912979\") " pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:40.030436 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:40.030389 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 16:38:40.030509 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:40.030454 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret podName:0f5cbd3c-3a54-4441-bea2-2643a5912979 nodeName:}" failed. No retries permitted until 2026-07-06 16:39:12.030438614 +0000 UTC m=+66.208297719 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret") pod "global-pull-secret-syncer-2m87r" (UID: "0f5cbd3c-3a54-4441-bea2-2643a5912979") : object "kube-system"/"original-pull-secret" not registered Jul 06 16:38:40.131452 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:40.131404 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9bd9\" (UniqueName: \"kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9\") pod \"network-check-target-xzfns\" (UID: \"52ebf132-7501-49f4-8615-28020de19a6f\") " pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:40.131659 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:40.131644 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 16:38:40.131719 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:40.131669 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 16:38:40.131719 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:40.131681 2580 projected.go:196] Error preparing data for projected volume kube-api-access-p9bd9 for pod openshift-network-diagnostics/network-check-target-xzfns: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:38:40.131817 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:40.131737 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9 podName:52ebf132-7501-49f4-8615-28020de19a6f nodeName:}" failed. No retries permitted until 2026-07-06 16:39:12.131721785 +0000 UTC m=+66.309580875 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-p9bd9" (UniqueName: "kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9") pod "network-check-target-xzfns" (UID: "52ebf132-7501-49f4-8615-28020de19a6f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 16:38:40.258664 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:40.258578 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:40.258814 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:40.258580 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:40.258814 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:40.258715 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m692" podUID="3f867a4a-dbf4-46a7-a79b-c30d916fa0f8" Jul 06 16:38:40.258814 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:40.258580 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:40.258814 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:40.258765 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzfns" podUID="52ebf132-7501-49f4-8615-28020de19a6f" Jul 06 16:38:40.259012 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:40.258830 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2m87r" podUID="0f5cbd3c-3a54-4441-bea2-2643a5912979" Jul 06 16:38:42.259242 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:42.259205 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:42.259777 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:42.259209 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:42.259777 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:42.259335 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m692" podUID="3f867a4a-dbf4-46a7-a79b-c30d916fa0f8" Jul 06 16:38:42.259777 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:42.259210 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:42.259777 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:42.259420 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2m87r" podUID="0f5cbd3c-3a54-4441-bea2-2643a5912979" Jul 06 16:38:42.259777 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:42.259514 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzfns" podUID="52ebf132-7501-49f4-8615-28020de19a6f" Jul 06 16:38:43.438923 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:43.438891 2580 generic.go:356] "Generic (PLEG): container finished" podID="6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb" containerID="b2c2754b1624d8cd14ea8830a9146820a7bccde79b261e833bb6dcdd8ff91916" exitCode=0 Jul 06 16:38:43.438923 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:43.438934 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44nqc" event={"ID":"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb","Type":"ContainerDied","Data":"b2c2754b1624d8cd14ea8830a9146820a7bccde79b261e833bb6dcdd8ff91916"} Jul 06 16:38:44.259025 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:44.258995 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:44.259025 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:44.259018 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:44.259225 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:44.258998 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:44.259225 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:44.259089 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xzfns" podUID="52ebf132-7501-49f4-8615-28020de19a6f" Jul 06 16:38:44.259225 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:44.259174 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2m87r" podUID="0f5cbd3c-3a54-4441-bea2-2643a5912979" Jul 06 16:38:44.259453 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:44.259240 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5m692" podUID="3f867a4a-dbf4-46a7-a79b-c30d916fa0f8" Jul 06 16:38:44.443906 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:44.443874 2580 generic.go:356] "Generic (PLEG): container finished" podID="6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb" containerID="62cc7349ad58563518a6100c54d53f2b4c976fca24ee717cbb2833dc7a8a1be2" exitCode=0 Jul 06 16:38:44.444259 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:44.443921 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44nqc" event={"ID":"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb","Type":"ContainerDied","Data":"62cc7349ad58563518a6100c54d53f2b4c976fca24ee717cbb2833dc7a8a1be2"} Jul 06 16:38:45.168041 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.168020 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-40.ec2.internal" event="NodeReady" Jul 06 16:38:45.168197 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.168131 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 16:38:45.232931 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.232896 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7c58c78ccb-5sns6"] Jul 06 16:38:45.247529 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.247501 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s"] Jul 06 16:38:45.247652 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.247616 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.255304 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.255279 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 16:38:45.255436 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.255325 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 16:38:45.255436 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.255356 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8rjds\"" Jul 06 16:38:45.255730 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.255697 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 16:38:45.263914 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.263893 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-cdh55"] Jul 06 16:38:45.263998 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.263987 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s" Jul 06 16:38:45.268027 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.268010 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 06 16:38:45.268961 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.268841 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dj8sn\"" Jul 06 16:38:45.268961 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.268856 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 06 16:38:45.268961 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.268843 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 16:38:45.268961 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.268884 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 16:38:45.269439 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.269422 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 16:38:45.281568 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.281546 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-87b54f4c7-4lpwt"] Jul 06 16:38:45.281655 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.281628 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" Jul 06 16:38:45.286986 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.286965 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 16:38:45.287268 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.287251 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-8v7cl\"" Jul 06 16:38:45.287850 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.287833 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 06 16:38:45.288431 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.288414 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 16:38:45.292540 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.292518 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 06 16:38:45.300403 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.300385 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 06 16:38:45.307609 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.307594 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c58c78ccb-5sns6"] Jul 06 16:38:45.307693 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.307621 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s"] Jul 06 16:38:45.307693 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.307637 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-mrznn"] Jul 06 16:38:45.307811 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.307725 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:45.311099 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.311078 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 06 16:38:45.311326 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.311311 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-szx98\"" Jul 06 16:38:45.311532 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.311517 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 06 16:38:45.311645 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.311603 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 06 16:38:45.311645 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.311603 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 16:38:45.311645 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.311646 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 06 16:38:45.312018 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.312005 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 16:38:45.327678 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.327657 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4xll7"] Jul 06 16:38:45.327829 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.327758 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mrznn" Jul 06 16:38:45.330129 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.330113 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 16:38:45.330203 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.330137 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 16:38:45.330972 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.330958 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-db7nl\"" Jul 06 16:38:45.345807 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.345787 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-cdh55"] Jul 06 16:38:45.345886 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.345814 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-87b54f4c7-4lpwt"] Jul 06 16:38:45.345886 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.345827 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4xll7"] Jul 06 16:38:45.345886 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.345840 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mrznn"] Jul 06 16:38:45.345973 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.345897 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4xll7" Jul 06 16:38:45.348681 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.348666 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 16:38:45.354308 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.354293 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 16:38:45.354389 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.354378 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 16:38:45.354427 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.354418 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-g8n7p\"" Jul 06 16:38:45.367422 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.367405 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/18464d2a-4b04-486f-bfc2-ac4d21f31578-ca-trust-extracted\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.367491 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.367451 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x247s\" (UID: \"59df778e-ece6-45e2-a0df-551df597ee79\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s" Jul 06 16:38:45.367533 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.367508 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbh85\" (UniqueName: \"kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-kube-api-access-sbh85\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.367533 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.367529 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns82s\" (UniqueName: \"kubernetes.io/projected/59df778e-ece6-45e2-a0df-551df597ee79-kube-api-access-ns82s\") pod \"cluster-monitoring-operator-64786d9cbb-x247s\" (UID: \"59df778e-ece6-45e2-a0df-551df597ee79\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s" Jul 06 16:38:45.367594 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.367558 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-certificates\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.367625 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.367616 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/18464d2a-4b04-486f-bfc2-ac4d21f31578-image-registry-private-configuration\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.367657 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.367633 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/18464d2a-4b04-486f-bfc2-ac4d21f31578-installation-pull-secrets\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.367657 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.367654 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/18464d2a-4b04-486f-bfc2-ac4d21f31578-trusted-ca\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.367717 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.367669 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-bound-sa-token\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.367717 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.367695 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.367787 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.367772 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/59df778e-ece6-45e2-a0df-551df597ee79-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-x247s\" (UID: \"59df778e-ece6-45e2-a0df-551df597ee79\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s" Jul 06 16:38:45.449481 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.449394 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-44nqc" event={"ID":"6ff8a8ec-3f37-4fc2-a436-8b1e9f1a48fb","Type":"ContainerStarted","Data":"4aa7b879fa532aace2bd0809a4fa47e3a2c3cdfbfc38e51091430752fda2ae75"} Jul 06 16:38:45.468015 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.467992 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbh85\" (UniqueName: \"kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-kube-api-access-sbh85\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.468134 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468023 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6aabdd3-9fd3-46e0-ac1b-9636787ae735-tmp\") pod \"insights-operator-5c847dcfc4-cdh55\" (UID: \"b6aabdd3-9fd3-46e0-ac1b-9636787ae735\") " pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" Jul 06 16:38:45.468134 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468040 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4bfd\" (UniqueName: \"kubernetes.io/projected/d0604265-3597-4501-9b25-8765c41d7f49-kube-api-access-j4bfd\") pod \"dns-default-mrznn\" (UID: \"d0604265-3597-4501-9b25-8765c41d7f49\") " pod="openshift-dns/dns-default-mrznn" Jul 06 16:38:45.468134 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468058 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0604265-3597-4501-9b25-8765c41d7f49-metrics-tls\") pod \"dns-default-mrznn\" (UID: \"d0604265-3597-4501-9b25-8765c41d7f49\") " pod="openshift-dns/dns-default-mrznn" Jul 06 16:38:45.468134 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468075 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt2gx\" (UniqueName: \"kubernetes.io/projected/f13c431e-6bae-4714-a1ba-49fa1c767244-kube-api-access-kt2gx\") pod \"ingress-canary-4xll7\" (UID: \"f13c431e-6bae-4714-a1ba-49fa1c767244\") " pod="openshift-ingress-canary/ingress-canary-4xll7" Jul 06 16:38:45.468134 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468115 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/18464d2a-4b04-486f-bfc2-ac4d21f31578-trusted-ca\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.468306 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468164 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d0604265-3597-4501-9b25-8765c41d7f49-config-volume\") pod \"dns-default-mrznn\" (UID: \"d0604265-3597-4501-9b25-8765c41d7f49\") " pod="openshift-dns/dns-default-mrznn" Jul 06 16:38:45.468306 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468259 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/59df778e-ece6-45e2-a0df-551df597ee79-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-x247s\" (UID: \"59df778e-ece6-45e2-a0df-551df597ee79\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s" Jul 06 16:38:45.468306 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468287 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-metrics-certs\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:45.468306 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468304 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b6aabdd3-9fd3-46e0-ac1b-9636787ae735-snapshots\") pod \"insights-operator-5c847dcfc4-cdh55\" (UID: \"b6aabdd3-9fd3-46e0-ac1b-9636787ae735\") " pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" Jul 06 16:38:45.468434 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468336 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x247s\" (UID: \"59df778e-ece6-45e2-a0df-551df597ee79\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s" Jul 06 16:38:45.468434 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468353 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0604265-3597-4501-9b25-8765c41d7f49-tmp-dir\") pod \"dns-default-mrznn\" (UID: \"d0604265-3597-4501-9b25-8765c41d7f49\") " pod="openshift-dns/dns-default-mrznn" Jul 06 16:38:45.468434 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468374 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-bound-sa-token\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.468434 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468394 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/18464d2a-4b04-486f-bfc2-ac4d21f31578-image-registry-private-configuration\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.468434 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468410 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/18464d2a-4b04-486f-bfc2-ac4d21f31578-installation-pull-secrets\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.468677 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468439 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b6aabdd3-9fd3-46e0-ac1b-9636787ae735-serving-cert\") pod \"insights-operator-5c847dcfc4-cdh55\" (UID: \"b6aabdd3-9fd3-46e0-ac1b-9636787ae735\") " pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" Jul 06 16:38:45.468677 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:45.468445 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 16:38:45.468677 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468484 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.468677 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:45.468528 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls podName:59df778e-ece6-45e2-a0df-551df597ee79 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:45.968506561 +0000 UTC m=+40.146365653 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-x247s" (UID: "59df778e-ece6-45e2-a0df-551df597ee79") : secret "cluster-monitoring-operator-tls" not found Jul 06 16:38:45.468677 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:45.468560 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 16:38:45.468677 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:45.468573 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c58c78ccb-5sns6: secret "image-registry-tls" not found Jul 06 16:38:45.468677 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:45.468624 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls podName:18464d2a-4b04-486f-bfc2-ac4d21f31578 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:45.968610048 +0000 UTC m=+40.146469140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls") pod "image-registry-7c58c78ccb-5sns6" (UID: "18464d2a-4b04-486f-bfc2-ac4d21f31578") : secret "image-registry-tls" not found Jul 06 16:38:45.468677 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468669 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4fxd\" (UniqueName: \"kubernetes.io/projected/b6aabdd3-9fd3-46e0-ac1b-9636787ae735-kube-api-access-p4fxd\") pod \"insights-operator-5c847dcfc4-cdh55\" (UID: \"b6aabdd3-9fd3-46e0-ac1b-9636787ae735\") " pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" Jul 06 16:38:45.469016 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468708 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-default-certificate\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:45.469016 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468743 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f13c431e-6bae-4714-a1ba-49fa1c767244-cert\") pod \"ingress-canary-4xll7\" (UID: \"f13c431e-6bae-4714-a1ba-49fa1c767244\") " pod="openshift-ingress-canary/ingress-canary-4xll7" Jul 06 16:38:45.469016 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468772 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/18464d2a-4b04-486f-bfc2-ac4d21f31578-ca-trust-extracted\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.469016 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468795 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9whwd\" (UniqueName: \"kubernetes.io/projected/f7221e3e-0a2d-4d91-b0f1-1b303255b302-kube-api-access-9whwd\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:45.469016 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468819 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6aabdd3-9fd3-46e0-ac1b-9636787ae735-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-cdh55\" (UID: \"b6aabdd3-9fd3-46e0-ac1b-9636787ae735\") " pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" Jul 06 16:38:45.469016 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468841 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:45.469016 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468879 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-certificates\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.469016 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468934 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-stats-auth\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:45.469016 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468963 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns82s\" (UniqueName: \"kubernetes.io/projected/59df778e-ece6-45e2-a0df-551df597ee79-kube-api-access-ns82s\") pod \"cluster-monitoring-operator-64786d9cbb-x247s\" (UID: \"59df778e-ece6-45e2-a0df-551df597ee79\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s" Jul 06 16:38:45.469016 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.468986 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6aabdd3-9fd3-46e0-ac1b-9636787ae735-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-cdh55\" (UID: \"b6aabdd3-9fd3-46e0-ac1b-9636787ae735\") " pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" Jul 06 16:38:45.469400 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.469055 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/59df778e-ece6-45e2-a0df-551df597ee79-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-x247s\" (UID: \"59df778e-ece6-45e2-a0df-551df597ee79\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s" Jul 06 16:38:45.469400 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.469075 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/18464d2a-4b04-486f-bfc2-ac4d21f31578-ca-trust-extracted\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.469400 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.469168 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/18464d2a-4b04-486f-bfc2-ac4d21f31578-trusted-ca\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.469400 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.469388 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-certificates\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.472383 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.472362 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/18464d2a-4b04-486f-bfc2-ac4d21f31578-image-registry-private-configuration\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.472489 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.472382 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/18464d2a-4b04-486f-bfc2-ac4d21f31578-installation-pull-secrets\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.478357 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.478338 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbh85\" (UniqueName: \"kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-kube-api-access-sbh85\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.490225 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.490199 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-bound-sa-token\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.491234 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.491181 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-44nqc" podStartSLOduration=10.204139395 podStartE2EDuration="39.491168111s" totalImagesPullingTime="29.287028716s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:38:06 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:38:45.478594269 +0000 UTC m=+39.656453381" watchObservedRunningTime="2026-07-06 16:38:45.491168111 +0000 UTC m=+39.669027245" Jul 06 16:38:45.493308 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.493290 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns82s\" (UniqueName: \"kubernetes.io/projected/59df778e-ece6-45e2-a0df-551df597ee79-kube-api-access-ns82s\") pod \"cluster-monitoring-operator-64786d9cbb-x247s\" (UID: \"59df778e-ece6-45e2-a0df-551df597ee79\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s" Jul 06 16:38:45.569743 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.569717 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b6aabdd3-9fd3-46e0-ac1b-9636787ae735-serving-cert\") pod \"insights-operator-5c847dcfc4-cdh55\" (UID: \"b6aabdd3-9fd3-46e0-ac1b-9636787ae735\") " pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" Jul 06 16:38:45.569819 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.569758 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4fxd\" (UniqueName: \"kubernetes.io/projected/b6aabdd3-9fd3-46e0-ac1b-9636787ae735-kube-api-access-p4fxd\") pod \"insights-operator-5c847dcfc4-cdh55\" (UID: \"b6aabdd3-9fd3-46e0-ac1b-9636787ae735\") " pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" Jul 06 16:38:45.569819 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.569779 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-default-certificate\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:45.569819 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.569805 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f13c431e-6bae-4714-a1ba-49fa1c767244-cert\") pod \"ingress-canary-4xll7\" (UID: \"f13c431e-6bae-4714-a1ba-49fa1c767244\") " pod="openshift-ingress-canary/ingress-canary-4xll7" Jul 06 16:38:45.569930 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.569829 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9whwd\" (UniqueName: \"kubernetes.io/projected/f7221e3e-0a2d-4d91-b0f1-1b303255b302-kube-api-access-9whwd\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:45.569930 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.569853 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6aabdd3-9fd3-46e0-ac1b-9636787ae735-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-cdh55\" (UID: \"b6aabdd3-9fd3-46e0-ac1b-9636787ae735\") " pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" Jul 06 16:38:45.569930 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.569874 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:45.570075 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.569929 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-stats-auth\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:45.570075 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.569956 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6aabdd3-9fd3-46e0-ac1b-9636787ae735-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-cdh55\" (UID: \"b6aabdd3-9fd3-46e0-ac1b-9636787ae735\") " pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" Jul 06 16:38:45.570075 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.569999 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6aabdd3-9fd3-46e0-ac1b-9636787ae735-tmp\") pod \"insights-operator-5c847dcfc4-cdh55\" (UID: \"b6aabdd3-9fd3-46e0-ac1b-9636787ae735\") " pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" Jul 06 16:38:45.570075 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.570022 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4bfd\" (UniqueName: \"kubernetes.io/projected/d0604265-3597-4501-9b25-8765c41d7f49-kube-api-access-j4bfd\") pod \"dns-default-mrznn\" (UID: \"d0604265-3597-4501-9b25-8765c41d7f49\") " pod="openshift-dns/dns-default-mrznn" Jul 06 16:38:45.570075 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.570067 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0604265-3597-4501-9b25-8765c41d7f49-metrics-tls\") pod \"dns-default-mrznn\" (UID: \"d0604265-3597-4501-9b25-8765c41d7f49\") " pod="openshift-dns/dns-default-mrznn" Jul 06 16:38:45.570308 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.570094 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt2gx\" (UniqueName: \"kubernetes.io/projected/f13c431e-6bae-4714-a1ba-49fa1c767244-kube-api-access-kt2gx\") pod \"ingress-canary-4xll7\" (UID: \"f13c431e-6bae-4714-a1ba-49fa1c767244\") " pod="openshift-ingress-canary/ingress-canary-4xll7" Jul 06 16:38:45.570308 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.570125 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d0604265-3597-4501-9b25-8765c41d7f49-config-volume\") pod \"dns-default-mrznn\" (UID: \"d0604265-3597-4501-9b25-8765c41d7f49\") " pod="openshift-dns/dns-default-mrznn" Jul 06 16:38:45.570308 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.570202 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-metrics-certs\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:45.570308 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:45.570241 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle podName:f7221e3e-0a2d-4d91-b0f1-1b303255b302 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:46.070222497 +0000 UTC m=+40.248081604 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle") pod "router-default-87b54f4c7-4lpwt" (UID: "f7221e3e-0a2d-4d91-b0f1-1b303255b302") : configmap references non-existent config key: service-ca.crt Jul 06 16:38:45.570308 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.570271 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b6aabdd3-9fd3-46e0-ac1b-9636787ae735-snapshots\") pod \"insights-operator-5c847dcfc4-cdh55\" (UID: \"b6aabdd3-9fd3-46e0-ac1b-9636787ae735\") " pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" Jul 06 16:38:45.570308 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:45.570287 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 16:38:45.570636 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.570320 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0604265-3597-4501-9b25-8765c41d7f49-tmp-dir\") pod \"dns-default-mrznn\" (UID: \"d0604265-3597-4501-9b25-8765c41d7f49\") " pod="openshift-dns/dns-default-mrznn" Jul 06 16:38:45.570636 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:45.570342 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-metrics-certs podName:f7221e3e-0a2d-4d91-b0f1-1b303255b302 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:46.070328477 +0000 UTC m=+40.248187587 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-metrics-certs") pod "router-default-87b54f4c7-4lpwt" (UID: "f7221e3e-0a2d-4d91-b0f1-1b303255b302") : secret "router-metrics-certs-default" not found Jul 06 16:38:45.570636 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.570609 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0604265-3597-4501-9b25-8765c41d7f49-tmp-dir\") pod \"dns-default-mrznn\" (UID: \"d0604265-3597-4501-9b25-8765c41d7f49\") " pod="openshift-dns/dns-default-mrznn" Jul 06 16:38:45.570826 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.570806 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6aabdd3-9fd3-46e0-ac1b-9636787ae735-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-cdh55\" (UID: \"b6aabdd3-9fd3-46e0-ac1b-9636787ae735\") " pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" Jul 06 16:38:45.570899 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.570876 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6aabdd3-9fd3-46e0-ac1b-9636787ae735-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-cdh55\" (UID: \"b6aabdd3-9fd3-46e0-ac1b-9636787ae735\") " pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" Jul 06 16:38:45.570989 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:45.570974 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 16:38:45.571046 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:45.571018 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0604265-3597-4501-9b25-8765c41d7f49-metrics-tls podName:d0604265-3597-4501-9b25-8765c41d7f49 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:46.071003746 +0000 UTC m=+40.248862843 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d0604265-3597-4501-9b25-8765c41d7f49-metrics-tls") pod "dns-default-mrznn" (UID: "d0604265-3597-4501-9b25-8765c41d7f49") : secret "dns-default-metrics-tls" not found Jul 06 16:38:45.571342 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.571318 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/b6aabdd3-9fd3-46e0-ac1b-9636787ae735-snapshots\") pod \"insights-operator-5c847dcfc4-cdh55\" (UID: \"b6aabdd3-9fd3-46e0-ac1b-9636787ae735\") " pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" Jul 06 16:38:45.571449 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.571431 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b6aabdd3-9fd3-46e0-ac1b-9636787ae735-tmp\") pod \"insights-operator-5c847dcfc4-cdh55\" (UID: \"b6aabdd3-9fd3-46e0-ac1b-9636787ae735\") " pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" Jul 06 16:38:45.571747 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.571728 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d0604265-3597-4501-9b25-8765c41d7f49-config-volume\") pod \"dns-default-mrznn\" (UID: \"d0604265-3597-4501-9b25-8765c41d7f49\") " pod="openshift-dns/dns-default-mrznn" Jul 06 16:38:45.572319 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.572297 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b6aabdd3-9fd3-46e0-ac1b-9636787ae735-serving-cert\") pod \"insights-operator-5c847dcfc4-cdh55\" (UID: \"b6aabdd3-9fd3-46e0-ac1b-9636787ae735\") " pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" Jul 06 16:38:45.572407 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.572358 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f13c431e-6bae-4714-a1ba-49fa1c767244-cert\") pod \"ingress-canary-4xll7\" (UID: \"f13c431e-6bae-4714-a1ba-49fa1c767244\") " pod="openshift-ingress-canary/ingress-canary-4xll7" Jul 06 16:38:45.572543 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.572526 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-default-certificate\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:45.572587 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.572547 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-stats-auth\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:45.590716 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.590699 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4fxd\" (UniqueName: \"kubernetes.io/projected/b6aabdd3-9fd3-46e0-ac1b-9636787ae735-kube-api-access-p4fxd\") pod \"insights-operator-5c847dcfc4-cdh55\" (UID: \"b6aabdd3-9fd3-46e0-ac1b-9636787ae735\") " pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" Jul 06 16:38:45.591228 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.591209 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9whwd\" (UniqueName: \"kubernetes.io/projected/f7221e3e-0a2d-4d91-b0f1-1b303255b302-kube-api-access-9whwd\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:45.599102 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.599084 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt2gx\" (UniqueName: \"kubernetes.io/projected/f13c431e-6bae-4714-a1ba-49fa1c767244-kube-api-access-kt2gx\") pod \"ingress-canary-4xll7\" (UID: \"f13c431e-6bae-4714-a1ba-49fa1c767244\") " pod="openshift-ingress-canary/ingress-canary-4xll7" Jul 06 16:38:45.605535 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.605517 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4bfd\" (UniqueName: \"kubernetes.io/projected/d0604265-3597-4501-9b25-8765c41d7f49-kube-api-access-j4bfd\") pod \"dns-default-mrznn\" (UID: \"d0604265-3597-4501-9b25-8765c41d7f49\") " pod="openshift-dns/dns-default-mrznn" Jul 06 16:38:45.653254 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.653233 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4xll7" Jul 06 16:38:45.689551 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.689522 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xqf97_66c85ad6-1024-423c-b7bc-2c17115326c1/konnectivity-agent/0.log" limit=1024 Jul 06 16:38:45.813238 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.813073 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-xqf97_66c85ad6-1024-423c-b7bc-2c17115326c1/konnectivity-agent/0.log" limit=1024 Jul 06 16:38:45.839871 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.839838 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4xll7"] Jul 06 16:38:45.843583 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:45.843560 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13c431e_6bae_4714_a1ba_49fa1c767244.slice/crio-586a04f52db8891cac416b7963f44969c9d0096156b45bf93d8509c078fe55aa WatchSource:0}: Error finding container 586a04f52db8891cac416b7963f44969c9d0096156b45bf93d8509c078fe55aa: Status 404 returned error can't find the container with id 586a04f52db8891cac416b7963f44969c9d0096156b45bf93d8509c078fe55aa Jul 06 16:38:45.889849 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.889826 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" Jul 06 16:38:45.974940 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.974912 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x247s\" (UID: \"59df778e-ece6-45e2-a0df-551df597ee79\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s" Jul 06 16:38:45.975072 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:45.974952 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:45.975072 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:45.975054 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 16:38:45.975196 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:45.975083 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 16:38:45.975196 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:45.975095 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c58c78ccb-5sns6: secret "image-registry-tls" not found Jul 06 16:38:45.975196 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:45.975121 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls podName:59df778e-ece6-45e2-a0df-551df597ee79 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:46.975100803 +0000 UTC m=+41.152959896 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-x247s" (UID: "59df778e-ece6-45e2-a0df-551df597ee79") : secret "cluster-monitoring-operator-tls" not found Jul 06 16:38:45.975196 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:45.975139 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls podName:18464d2a-4b04-486f-bfc2-ac4d21f31578 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:46.975130375 +0000 UTC m=+41.152989466 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls") pod "image-registry-7c58c78ccb-5sns6" (UID: "18464d2a-4b04-486f-bfc2-ac4d21f31578") : secret "image-registry-tls" not found Jul 06 16:38:46.016497 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:46.016385 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-cdh55"] Jul 06 16:38:46.019203 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:46.019179 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6aabdd3_9fd3_46e0_ac1b_9636787ae735.slice/crio-b55156379d20668e26c2b9081a39c4f3ddde0d740c82f7c1909caa5bfc254a6a WatchSource:0}: Error finding container b55156379d20668e26c2b9081a39c4f3ddde0d740c82f7c1909caa5bfc254a6a: Status 404 returned error can't find the container with id b55156379d20668e26c2b9081a39c4f3ddde0d740c82f7c1909caa5bfc254a6a Jul 06 16:38:46.075973 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:46.075943 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:46.076063 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:46.075995 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0604265-3597-4501-9b25-8765c41d7f49-metrics-tls\") pod \"dns-default-mrznn\" (UID: \"d0604265-3597-4501-9b25-8765c41d7f49\") " pod="openshift-dns/dns-default-mrznn" Jul 06 16:38:46.076063 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:46.076037 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-metrics-certs\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:46.076227 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:46.076131 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle podName:f7221e3e-0a2d-4d91-b0f1-1b303255b302 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:47.076106819 +0000 UTC m=+41.253965915 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle") pod "router-default-87b54f4c7-4lpwt" (UID: "f7221e3e-0a2d-4d91-b0f1-1b303255b302") : configmap references non-existent config key: service-ca.crt Jul 06 16:38:46.076227 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:46.076151 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 16:38:46.076227 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:46.076192 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0604265-3597-4501-9b25-8765c41d7f49-metrics-tls podName:d0604265-3597-4501-9b25-8765c41d7f49 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:47.076181465 +0000 UTC m=+41.254040559 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d0604265-3597-4501-9b25-8765c41d7f49-metrics-tls") pod "dns-default-mrznn" (UID: "d0604265-3597-4501-9b25-8765c41d7f49") : secret "dns-default-metrics-tls" not found Jul 06 16:38:46.076227 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:46.076151 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 16:38:46.076227 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:46.076221 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-metrics-certs podName:f7221e3e-0a2d-4d91-b0f1-1b303255b302 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:47.076213223 +0000 UTC m=+41.254072313 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-metrics-certs") pod "router-default-87b54f4c7-4lpwt" (UID: "f7221e3e-0a2d-4d91-b0f1-1b303255b302") : secret "router-metrics-certs-default" not found Jul 06 16:38:46.259407 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:46.259368 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:38:46.259593 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:46.259493 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:38:46.259593 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:46.259493 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:38:46.262465 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:46.262449 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 16:38:46.262842 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:46.262815 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 16:38:46.262974 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:46.262951 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ntdtl\"" Jul 06 16:38:46.263088 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:46.263025 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 16:38:46.263482 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:46.263450 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q4r89\"" Jul 06 16:38:46.263557 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:46.263480 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 16:38:46.453046 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:46.453008 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" event={"ID":"b6aabdd3-9fd3-46e0-ac1b-9636787ae735","Type":"ContainerStarted","Data":"b55156379d20668e26c2b9081a39c4f3ddde0d740c82f7c1909caa5bfc254a6a"} Jul 06 16:38:46.454005 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:46.453982 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4xll7" event={"ID":"f13c431e-6bae-4714-a1ba-49fa1c767244","Type":"ContainerStarted","Data":"586a04f52db8891cac416b7963f44969c9d0096156b45bf93d8509c078fe55aa"} Jul 06 16:38:46.982711 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:46.982672 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:46.982887 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:46.982827 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x247s\" (UID: \"59df778e-ece6-45e2-a0df-551df597ee79\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s" Jul 06 16:38:46.982887 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:46.982851 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 16:38:46.982887 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:46.982875 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c58c78ccb-5sns6: secret "image-registry-tls" not found Jul 06 16:38:46.983039 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:46.982947 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls podName:18464d2a-4b04-486f-bfc2-ac4d21f31578 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:48.982925674 +0000 UTC m=+43.160784777 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls") pod "image-registry-7c58c78ccb-5sns6" (UID: "18464d2a-4b04-486f-bfc2-ac4d21f31578") : secret "image-registry-tls" not found Jul 06 16:38:46.983039 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:46.982951 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 16:38:46.983039 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:46.983004 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls podName:59df778e-ece6-45e2-a0df-551df597ee79 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:48.982987885 +0000 UTC m=+43.160846988 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-x247s" (UID: "59df778e-ece6-45e2-a0df-551df597ee79") : secret "cluster-monitoring-operator-tls" not found Jul 06 16:38:47.083746 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:47.083674 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:47.083926 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:47.083766 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0604265-3597-4501-9b25-8765c41d7f49-metrics-tls\") pod \"dns-default-mrznn\" (UID: \"d0604265-3597-4501-9b25-8765c41d7f49\") " pod="openshift-dns/dns-default-mrznn" Jul 06 16:38:47.083926 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:47.083833 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-metrics-certs\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:47.083926 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:47.083892 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle podName:f7221e3e-0a2d-4d91-b0f1-1b303255b302 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:49.083867734 +0000 UTC m=+43.261726857 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle") pod "router-default-87b54f4c7-4lpwt" (UID: "f7221e3e-0a2d-4d91-b0f1-1b303255b302") : configmap references non-existent config key: service-ca.crt Jul 06 16:38:47.084100 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:47.083928 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 16:38:47.084100 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:47.083926 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 16:38:47.084100 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:47.083976 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-metrics-certs podName:f7221e3e-0a2d-4d91-b0f1-1b303255b302 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:49.083960426 +0000 UTC m=+43.261819517 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-metrics-certs") pod "router-default-87b54f4c7-4lpwt" (UID: "f7221e3e-0a2d-4d91-b0f1-1b303255b302") : secret "router-metrics-certs-default" not found Jul 06 16:38:47.084100 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:47.084008 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0604265-3597-4501-9b25-8765c41d7f49-metrics-tls podName:d0604265-3597-4501-9b25-8765c41d7f49 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:49.083998459 +0000 UTC m=+43.261857559 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d0604265-3597-4501-9b25-8765c41d7f49-metrics-tls") pod "dns-default-mrznn" (UID: "d0604265-3597-4501-9b25-8765c41d7f49") : secret "dns-default-metrics-tls" not found Jul 06 16:38:49.002205 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:49.001976 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x247s\" (UID: \"59df778e-ece6-45e2-a0df-551df597ee79\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s" Jul 06 16:38:49.002628 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:49.002241 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:49.002628 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:49.002131 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 16:38:49.002628 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:49.002342 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 16:38:49.002628 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:49.002355 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c58c78ccb-5sns6: secret "image-registry-tls" not found Jul 06 16:38:49.002628 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:49.002346 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls podName:59df778e-ece6-45e2-a0df-551df597ee79 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:53.002327555 +0000 UTC m=+47.180186661 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-x247s" (UID: "59df778e-ece6-45e2-a0df-551df597ee79") : secret "cluster-monitoring-operator-tls" not found Jul 06 16:38:49.002628 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:49.002396 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls podName:18464d2a-4b04-486f-bfc2-ac4d21f31578 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:53.00238511 +0000 UTC m=+47.180244200 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls") pod "image-registry-7c58c78ccb-5sns6" (UID: "18464d2a-4b04-486f-bfc2-ac4d21f31578") : secret "image-registry-tls" not found Jul 06 16:38:49.103383 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:49.103350 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:49.103588 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:49.103422 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0604265-3597-4501-9b25-8765c41d7f49-metrics-tls\") pod \"dns-default-mrznn\" (UID: \"d0604265-3597-4501-9b25-8765c41d7f49\") " pod="openshift-dns/dns-default-mrznn" Jul 06 16:38:49.103588 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:49.103477 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-metrics-certs\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:49.103588 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:49.103556 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle podName:f7221e3e-0a2d-4d91-b0f1-1b303255b302 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:53.103534131 +0000 UTC m=+47.281393225 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle") pod "router-default-87b54f4c7-4lpwt" (UID: "f7221e3e-0a2d-4d91-b0f1-1b303255b302") : configmap references non-existent config key: service-ca.crt Jul 06 16:38:49.103588 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:49.103580 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 16:38:49.103588 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:49.103585 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 16:38:49.103828 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:49.103627 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-metrics-certs podName:f7221e3e-0a2d-4d91-b0f1-1b303255b302 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:53.103616485 +0000 UTC m=+47.281475574 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-metrics-certs") pod "router-default-87b54f4c7-4lpwt" (UID: "f7221e3e-0a2d-4d91-b0f1-1b303255b302") : secret "router-metrics-certs-default" not found Jul 06 16:38:49.103828 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:49.103643 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0604265-3597-4501-9b25-8765c41d7f49-metrics-tls podName:d0604265-3597-4501-9b25-8765c41d7f49 nodeName:}" failed. No retries permitted until 2026-07-06 16:38:53.103634786 +0000 UTC m=+47.281493875 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d0604265-3597-4501-9b25-8765c41d7f49-metrics-tls") pod "dns-default-mrznn" (UID: "d0604265-3597-4501-9b25-8765c41d7f49") : secret "dns-default-metrics-tls" not found Jul 06 16:38:50.465110 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:50.465074 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" event={"ID":"b6aabdd3-9fd3-46e0-ac1b-9636787ae735","Type":"ContainerStarted","Data":"746c1a9d05da6e4e68a846e91a0b693eade0cf598bc6d08061fb765d89719b42"} Jul 06 16:38:50.466671 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:50.466646 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4xll7" event={"ID":"f13c431e-6bae-4714-a1ba-49fa1c767244","Type":"ContainerStarted","Data":"c1c62458729a79e9f7eb021dc686897c69e12471a63bad9212e1fa2fe1d982a5"} Jul 06 16:38:50.493407 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:50.493363 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" podStartSLOduration=2.609136386 podStartE2EDuration="6.493348432s" totalImagesPullingTime="3.884212046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:38:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:38:50.492915039 +0000 UTC m=+44.670774152" watchObservedRunningTime="2026-07-06 16:38:50.493348432 +0000 UTC m=+44.671207554" Jul 06 16:38:50.532854 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:50.532795 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4xll7" podStartSLOduration=1.466726783 podStartE2EDuration="5.532778296s" totalImagesPullingTime="4.066051513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:38:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:38:50.532203204 +0000 UTC m=+44.710062329" watchObservedRunningTime="2026-07-06 16:38:50.532778296 +0000 UTC m=+44.710637425" Jul 06 16:38:50.817493 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:50.817381 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-ps5hn"] Jul 06 16:38:50.821896 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:50.821873 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:50.823920 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:50.823900 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 16:38:50.824028 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:50.823934 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 16:38:50.824170 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:50.824158 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nl5jr\"" Jul 06 16:38:50.854068 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:50.854049 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ps5hn"] Jul 06 16:38:50.918379 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:50.918343 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:50.918538 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:50.918396 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-crio-socket\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:50.918538 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:50.918465 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:50.918538 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:50.918520 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-data-volume\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:50.918645 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:50.918546 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jqz7\" (UniqueName: \"kubernetes.io/projected/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-kube-api-access-8jqz7\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:51.019030 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:51.019000 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:51.019193 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:51.019041 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-crio-socket\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:51.019193 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:51.019090 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:51.019193 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:51.019116 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-data-volume\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:51.019193 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:51.019128 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 16:38:51.019337 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:51.019132 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jqz7\" (UniqueName: \"kubernetes.io/projected/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-kube-api-access-8jqz7\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:51.019337 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:51.019213 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-kube-rbac-proxy-cm podName:6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e nodeName:}" failed. No retries permitted until 2026-07-06 16:38:51.519191962 +0000 UTC m=+45.697051066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ps5hn" (UID: "6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e") : configmap "kube-rbac-proxy" not found Jul 06 16:38:51.019337 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:51.019226 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 16:38:51.019337 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:51.019278 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-insights-runtime-extractor-tls podName:6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e nodeName:}" failed. No retries permitted until 2026-07-06 16:38:51.51926319 +0000 UTC m=+45.697122286 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ps5hn" (UID: "6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e") : secret "insights-runtime-extractor-tls" not found Jul 06 16:38:51.019463 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:51.019376 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-crio-socket\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:51.019528 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:51.019508 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-data-volume\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:51.039500 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:51.039458 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jqz7\" (UniqueName: \"kubernetes.io/projected/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-kube-api-access-8jqz7\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:51.523547 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:51.523502 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:51.524114 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:51.523629 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:51.524114 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:51.523633 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 16:38:51.524114 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:51.523712 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-insights-runtime-extractor-tls podName:6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e nodeName:}" failed. No retries permitted until 2026-07-06 16:38:52.523693693 +0000 UTC m=+46.701552794 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ps5hn" (UID: "6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e") : secret "insights-runtime-extractor-tls" not found Jul 06 16:38:51.524114 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:51.523726 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 16:38:51.524114 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:51.523805 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-kube-rbac-proxy-cm podName:6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e nodeName:}" failed. No retries permitted until 2026-07-06 16:38:52.523776974 +0000 UTC m=+46.701636064 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ps5hn" (UID: "6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e") : configmap "kube-rbac-proxy" not found Jul 06 16:38:52.178008 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:52.177976 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q4flx_cf835c0c-f2ea-416f-a27f-619d7eeaf9c6/dns-node-resolver/0.log" Jul 06 16:38:52.531721 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:52.531637 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:52.531721 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:52.531711 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:52.532114 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:52.531762 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 16:38:52.532114 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:52.531845 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-kube-rbac-proxy-cm podName:6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e nodeName:}" failed. No retries permitted until 2026-07-06 16:38:54.53182759 +0000 UTC m=+48.709686682 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ps5hn" (UID: "6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e") : configmap "kube-rbac-proxy" not found Jul 06 16:38:52.532114 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:52.531845 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 16:38:52.532114 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:52.531874 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-insights-runtime-extractor-tls podName:6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e nodeName:}" failed. No retries permitted until 2026-07-06 16:38:54.531868562 +0000 UTC m=+48.709727652 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ps5hn" (UID: "6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e") : secret "insights-runtime-extractor-tls" not found Jul 06 16:38:52.978026 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:52.977992 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zw578_206b0067-d25e-40fd-ab2d-5a834df0c969/node-ca/0.log" Jul 06 16:38:53.036395 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.036354 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x247s\" (UID: \"59df778e-ece6-45e2-a0df-551df597ee79\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s" Jul 06 16:38:53.036560 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.036408 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:38:53.036560 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:53.036531 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 16:38:53.036643 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:53.036556 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 16:38:53.036643 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:53.036572 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c58c78ccb-5sns6: secret "image-registry-tls" not found Jul 06 16:38:53.036643 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:53.036606 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls podName:59df778e-ece6-45e2-a0df-551df597ee79 nodeName:}" failed. No retries permitted until 2026-07-06 16:39:01.036587457 +0000 UTC m=+55.214446547 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-x247s" (UID: "59df778e-ece6-45e2-a0df-551df597ee79") : secret "cluster-monitoring-operator-tls" not found Jul 06 16:38:53.036643 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:53.036628 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls podName:18464d2a-4b04-486f-bfc2-ac4d21f31578 nodeName:}" failed. No retries permitted until 2026-07-06 16:39:01.036616938 +0000 UTC m=+55.214476031 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls") pod "image-registry-7c58c78ccb-5sns6" (UID: "18464d2a-4b04-486f-bfc2-ac4d21f31578") : secret "image-registry-tls" not found Jul 06 16:38:53.137464 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.137424 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-metrics-certs\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:53.137671 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.137552 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:38:53.137671 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:53.137586 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 16:38:53.137671 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.137616 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0604265-3597-4501-9b25-8765c41d7f49-metrics-tls\") pod \"dns-default-mrznn\" (UID: \"d0604265-3597-4501-9b25-8765c41d7f49\") " pod="openshift-dns/dns-default-mrznn" Jul 06 16:38:53.137671 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:53.137650 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-metrics-certs podName:f7221e3e-0a2d-4d91-b0f1-1b303255b302 nodeName:}" failed. No retries permitted until 2026-07-06 16:39:01.137634581 +0000 UTC m=+55.315493674 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-metrics-certs") pod "router-default-87b54f4c7-4lpwt" (UID: "f7221e3e-0a2d-4d91-b0f1-1b303255b302") : secret "router-metrics-certs-default" not found Jul 06 16:38:53.137857 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:53.137696 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle podName:f7221e3e-0a2d-4d91-b0f1-1b303255b302 nodeName:}" failed. No retries permitted until 2026-07-06 16:39:01.137682356 +0000 UTC m=+55.315541447 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle") pod "router-default-87b54f4c7-4lpwt" (UID: "f7221e3e-0a2d-4d91-b0f1-1b303255b302") : configmap references non-existent config key: service-ca.crt Jul 06 16:38:53.137857 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:53.137704 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 16:38:53.137857 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:53.137754 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0604265-3597-4501-9b25-8765c41d7f49-metrics-tls podName:d0604265-3597-4501-9b25-8765c41d7f49 nodeName:}" failed. No retries permitted until 2026-07-06 16:39:01.137741418 +0000 UTC m=+55.315600521 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d0604265-3597-4501-9b25-8765c41d7f49-metrics-tls") pod "dns-default-mrznn" (UID: "d0604265-3597-4501-9b25-8765c41d7f49") : secret "dns-default-metrics-tls" not found Jul 06 16:38:53.183257 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.183220 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht4xk"] Jul 06 16:38:53.217222 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.217192 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mct68"] Jul 06 16:38:53.217350 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.217307 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht4xk" Jul 06 16:38:53.220721 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.220683 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 06 16:38:53.220829 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.220749 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 06 16:38:53.220887 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.220877 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 06 16:38:53.220957 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.220943 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6zmpk\"" Jul 06 16:38:53.231246 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.231192 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mct68"] Jul 06 16:38:53.231246 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.231225 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht4xk"] Jul 06 16:38:53.231391 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.231311 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mct68" Jul 06 16:38:53.235288 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.235263 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 06 16:38:53.235288 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.235266 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 06 16:38:53.235435 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.235417 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-zr489\"" Jul 06 16:38:53.238126 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.238105 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcgzj\" (UniqueName: \"kubernetes.io/projected/5d998ace-6b9c-435e-9360-b0776df34e66-kube-api-access-qcgzj\") pod \"volume-data-source-validator-5d456fdbcd-mct68\" (UID: \"5d998ace-6b9c-435e-9360-b0776df34e66\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mct68" Jul 06 16:38:53.238273 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.238141 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/36d6010c-f9cd-4fea-9e77-3cf687d3616b-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ht4xk\" (UID: \"36d6010c-f9cd-4fea-9e77-3cf687d3616b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht4xk" Jul 06 16:38:53.238273 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.238166 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7qv2\" (UniqueName: \"kubernetes.io/projected/36d6010c-f9cd-4fea-9e77-3cf687d3616b-kube-api-access-w7qv2\") pod \"cluster-samples-operator-56fcf86bf-ht4xk\" (UID: \"36d6010c-f9cd-4fea-9e77-3cf687d3616b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht4xk" Jul 06 16:38:53.339058 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.338953 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcgzj\" (UniqueName: \"kubernetes.io/projected/5d998ace-6b9c-435e-9360-b0776df34e66-kube-api-access-qcgzj\") pod \"volume-data-source-validator-5d456fdbcd-mct68\" (UID: \"5d998ace-6b9c-435e-9360-b0776df34e66\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mct68" Jul 06 16:38:53.339058 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.338996 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/36d6010c-f9cd-4fea-9e77-3cf687d3616b-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ht4xk\" (UID: \"36d6010c-f9cd-4fea-9e77-3cf687d3616b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht4xk" Jul 06 16:38:53.339058 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.339019 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7qv2\" (UniqueName: \"kubernetes.io/projected/36d6010c-f9cd-4fea-9e77-3cf687d3616b-kube-api-access-w7qv2\") pod \"cluster-samples-operator-56fcf86bf-ht4xk\" (UID: \"36d6010c-f9cd-4fea-9e77-3cf687d3616b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht4xk" Jul 06 16:38:53.339672 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:53.339652 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 16:38:53.339977 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:53.339949 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36d6010c-f9cd-4fea-9e77-3cf687d3616b-samples-operator-tls podName:36d6010c-f9cd-4fea-9e77-3cf687d3616b nodeName:}" failed. No retries permitted until 2026-07-06 16:38:53.839926498 +0000 UTC m=+48.017785589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/36d6010c-f9cd-4fea-9e77-3cf687d3616b-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-ht4xk" (UID: "36d6010c-f9cd-4fea-9e77-3cf687d3616b") : secret "samples-operator-tls" not found Jul 06 16:38:53.362206 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.361761 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcgzj\" (UniqueName: \"kubernetes.io/projected/5d998ace-6b9c-435e-9360-b0776df34e66-kube-api-access-qcgzj\") pod \"volume-data-source-validator-5d456fdbcd-mct68\" (UID: \"5d998ace-6b9c-435e-9360-b0776df34e66\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mct68" Jul 06 16:38:53.365867 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.365813 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7qv2\" (UniqueName: \"kubernetes.io/projected/36d6010c-f9cd-4fea-9e77-3cf687d3616b-kube-api-access-w7qv2\") pod \"cluster-samples-operator-56fcf86bf-ht4xk\" (UID: \"36d6010c-f9cd-4fea-9e77-3cf687d3616b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht4xk" Jul 06 16:38:53.375506 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.375457 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4xll7_f13c431e-6bae-4714-a1ba-49fa1c767244/serve-healthcheck-canary/0.log" Jul 06 16:38:53.539335 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.539260 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mct68" Jul 06 16:38:53.681566 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.681536 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mct68"] Jul 06 16:38:53.684482 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:53.684436 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d998ace_6b9c_435e_9360_b0776df34e66.slice/crio-c907845088cf45c59c36de915fe90c99f330e42c14d20a1f515031661f8079ca WatchSource:0}: Error finding container c907845088cf45c59c36de915fe90c99f330e42c14d20a1f515031661f8079ca: Status 404 returned error can't find the container with id c907845088cf45c59c36de915fe90c99f330e42c14d20a1f515031661f8079ca Jul 06 16:38:53.842363 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:53.842339 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/36d6010c-f9cd-4fea-9e77-3cf687d3616b-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ht4xk\" (UID: \"36d6010c-f9cd-4fea-9e77-3cf687d3616b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht4xk" Jul 06 16:38:53.842511 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:53.842462 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 16:38:53.842564 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:53.842541 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36d6010c-f9cd-4fea-9e77-3cf687d3616b-samples-operator-tls podName:36d6010c-f9cd-4fea-9e77-3cf687d3616b nodeName:}" failed. No retries permitted until 2026-07-06 16:38:54.84252685 +0000 UTC m=+49.020385939 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/36d6010c-f9cd-4fea-9e77-3cf687d3616b-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-ht4xk" (UID: "36d6010c-f9cd-4fea-9e77-3cf687d3616b") : secret "samples-operator-tls" not found Jul 06 16:38:54.194906 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.194824 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-7dtv6"] Jul 06 16:38:54.227005 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.226981 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-7dtv6" Jul 06 16:38:54.232078 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.232054 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-7dtv6"] Jul 06 16:38:54.235908 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.235887 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 06 16:38:54.236026 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.235986 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 06 16:38:54.236026 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.235990 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 06 16:38:54.236026 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.236008 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-v8nj2\"" Jul 06 16:38:54.246583 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.246562 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpxrk\" (UniqueName: \"kubernetes.io/projected/861b3db1-e952-457e-abdd-d36829821c19-kube-api-access-tpxrk\") pod \"kube-storage-version-migrator-operator-846bf668f7-7dtv6\" (UID: \"861b3db1-e952-457e-abdd-d36829821c19\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-7dtv6" Jul 06 16:38:54.246673 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.246592 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/861b3db1-e952-457e-abdd-d36829821c19-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-7dtv6\" (UID: \"861b3db1-e952-457e-abdd-d36829821c19\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-7dtv6" Jul 06 16:38:54.246751 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.246735 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/861b3db1-e952-457e-abdd-d36829821c19-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-7dtv6\" (UID: \"861b3db1-e952-457e-abdd-d36829821c19\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-7dtv6" Jul 06 16:38:54.254045 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.254030 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 06 16:38:54.348044 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.348010 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/861b3db1-e952-457e-abdd-d36829821c19-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-7dtv6\" (UID: \"861b3db1-e952-457e-abdd-d36829821c19\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-7dtv6" Jul 06 16:38:54.348224 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.348209 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/861b3db1-e952-457e-abdd-d36829821c19-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-7dtv6\" (UID: \"861b3db1-e952-457e-abdd-d36829821c19\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-7dtv6" Jul 06 16:38:54.348279 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.348260 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpxrk\" (UniqueName: \"kubernetes.io/projected/861b3db1-e952-457e-abdd-d36829821c19-kube-api-access-tpxrk\") pod \"kube-storage-version-migrator-operator-846bf668f7-7dtv6\" (UID: \"861b3db1-e952-457e-abdd-d36829821c19\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-7dtv6" Jul 06 16:38:54.348804 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.348785 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/861b3db1-e952-457e-abdd-d36829821c19-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-7dtv6\" (UID: \"861b3db1-e952-457e-abdd-d36829821c19\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-7dtv6" Jul 06 16:38:54.350305 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.350286 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/861b3db1-e952-457e-abdd-d36829821c19-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-7dtv6\" (UID: \"861b3db1-e952-457e-abdd-d36829821c19\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-7dtv6" Jul 06 16:38:54.369479 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.369438 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpxrk\" (UniqueName: \"kubernetes.io/projected/861b3db1-e952-457e-abdd-d36829821c19-kube-api-access-tpxrk\") pod \"kube-storage-version-migrator-operator-846bf668f7-7dtv6\" (UID: \"861b3db1-e952-457e-abdd-d36829821c19\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-7dtv6" Jul 06 16:38:54.439557 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.439528 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-pth6b"] Jul 06 16:38:54.460602 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.460552 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-pth6b" Jul 06 16:38:54.464755 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.464729 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4ggmp"] Jul 06 16:38:54.466383 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.466357 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 06 16:38:54.466527 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.466411 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 06 16:38:54.467080 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.467060 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 06 16:38:54.467194 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.467177 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-7p4mv\"" Jul 06 16:38:54.469000 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.468967 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 06 16:38:54.475801 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.475776 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 06 16:38:54.486630 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.486615 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-pth6b"] Jul 06 16:38:54.486714 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.486641 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mct68" event={"ID":"5d998ace-6b9c-435e-9360-b0776df34e66","Type":"ContainerStarted","Data":"c907845088cf45c59c36de915fe90c99f330e42c14d20a1f515031661f8079ca"} Jul 06 16:38:54.486714 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.486712 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4ggmp" Jul 06 16:38:54.490213 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.490178 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 06 16:38:54.490765 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.490753 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 06 16:38:54.490925 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.490908 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-wmbqz\"" Jul 06 16:38:54.491207 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.491191 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4ggmp"] Jul 06 16:38:54.495276 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.495258 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 06 16:38:54.497677 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.497661 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 06 16:38:54.535395 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.535367 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-7dtv6" Jul 06 16:38:54.549831 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.549806 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2372b154-c850-4003-ba8a-3534dd9c8cde-config\") pod \"service-ca-operator-5d46d9ffd4-4ggmp\" (UID: \"2372b154-c850-4003-ba8a-3534dd9c8cde\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4ggmp" Jul 06 16:38:54.550166 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.549848 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9snlc\" (UniqueName: \"kubernetes.io/projected/44d046d2-51c5-4769-a665-1805b2c28340-kube-api-access-9snlc\") pod \"console-operator-68b757fb67-pth6b\" (UID: \"44d046d2-51c5-4769-a665-1805b2c28340\") " pod="openshift-console-operator/console-operator-68b757fb67-pth6b" Jul 06 16:38:54.550166 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.549869 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2372b154-c850-4003-ba8a-3534dd9c8cde-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-4ggmp\" (UID: \"2372b154-c850-4003-ba8a-3534dd9c8cde\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4ggmp" Jul 06 16:38:54.550166 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.549935 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:54.550166 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:54.549988 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 16:38:54.550166 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.550007 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/44d046d2-51c5-4769-a665-1805b2c28340-trusted-ca\") pod \"console-operator-68b757fb67-pth6b\" (UID: \"44d046d2-51c5-4769-a665-1805b2c28340\") " pod="openshift-console-operator/console-operator-68b757fb67-pth6b" Jul 06 16:38:54.550166 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:54.550030 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-kube-rbac-proxy-cm podName:6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e nodeName:}" failed. No retries permitted until 2026-07-06 16:38:58.55001707 +0000 UTC m=+52.727876161 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ps5hn" (UID: "6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e") : configmap "kube-rbac-proxy" not found Jul 06 16:38:54.550166 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.550134 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jp9x2\" (UniqueName: \"kubernetes.io/projected/2372b154-c850-4003-ba8a-3534dd9c8cde-kube-api-access-jp9x2\") pod \"service-ca-operator-5d46d9ffd4-4ggmp\" (UID: \"2372b154-c850-4003-ba8a-3534dd9c8cde\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4ggmp" Jul 06 16:38:54.550166 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.550162 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:54.550455 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.550193 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/44d046d2-51c5-4769-a665-1805b2c28340-config\") pod \"console-operator-68b757fb67-pth6b\" (UID: \"44d046d2-51c5-4769-a665-1805b2c28340\") " pod="openshift-console-operator/console-operator-68b757fb67-pth6b" Jul 06 16:38:54.550455 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.550213 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/44d046d2-51c5-4769-a665-1805b2c28340-serving-cert\") pod \"console-operator-68b757fb67-pth6b\" (UID: \"44d046d2-51c5-4769-a665-1805b2c28340\") " pod="openshift-console-operator/console-operator-68b757fb67-pth6b" Jul 06 16:38:54.550455 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:54.550292 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 16:38:54.550455 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:54.550331 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-insights-runtime-extractor-tls podName:6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e nodeName:}" failed. No retries permitted until 2026-07-06 16:38:58.550320684 +0000 UTC m=+52.728179777 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ps5hn" (UID: "6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e") : secret "insights-runtime-extractor-tls" not found Jul 06 16:38:54.651555 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.651526 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/44d046d2-51c5-4769-a665-1805b2c28340-config\") pod \"console-operator-68b757fb67-pth6b\" (UID: \"44d046d2-51c5-4769-a665-1805b2c28340\") " pod="openshift-console-operator/console-operator-68b757fb67-pth6b" Jul 06 16:38:54.651671 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.651570 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/44d046d2-51c5-4769-a665-1805b2c28340-serving-cert\") pod \"console-operator-68b757fb67-pth6b\" (UID: \"44d046d2-51c5-4769-a665-1805b2c28340\") " pod="openshift-console-operator/console-operator-68b757fb67-pth6b" Jul 06 16:38:54.651671 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.651622 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2372b154-c850-4003-ba8a-3534dd9c8cde-config\") pod \"service-ca-operator-5d46d9ffd4-4ggmp\" (UID: \"2372b154-c850-4003-ba8a-3534dd9c8cde\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4ggmp" Jul 06 16:38:54.651740 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.651674 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9snlc\" (UniqueName: \"kubernetes.io/projected/44d046d2-51c5-4769-a665-1805b2c28340-kube-api-access-9snlc\") pod \"console-operator-68b757fb67-pth6b\" (UID: \"44d046d2-51c5-4769-a665-1805b2c28340\") " pod="openshift-console-operator/console-operator-68b757fb67-pth6b" Jul 06 16:38:54.651740 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.651712 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2372b154-c850-4003-ba8a-3534dd9c8cde-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-4ggmp\" (UID: \"2372b154-c850-4003-ba8a-3534dd9c8cde\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4ggmp" Jul 06 16:38:54.653525 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.652094 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/44d046d2-51c5-4769-a665-1805b2c28340-trusted-ca\") pod \"console-operator-68b757fb67-pth6b\" (UID: \"44d046d2-51c5-4769-a665-1805b2c28340\") " pod="openshift-console-operator/console-operator-68b757fb67-pth6b" Jul 06 16:38:54.653525 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.652631 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2372b154-c850-4003-ba8a-3534dd9c8cde-config\") pod \"service-ca-operator-5d46d9ffd4-4ggmp\" (UID: \"2372b154-c850-4003-ba8a-3534dd9c8cde\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4ggmp" Jul 06 16:38:54.653525 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.653401 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jp9x2\" (UniqueName: \"kubernetes.io/projected/2372b154-c850-4003-ba8a-3534dd9c8cde-kube-api-access-jp9x2\") pod \"service-ca-operator-5d46d9ffd4-4ggmp\" (UID: \"2372b154-c850-4003-ba8a-3534dd9c8cde\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4ggmp" Jul 06 16:38:54.653525 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.653456 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/44d046d2-51c5-4769-a665-1805b2c28340-config\") pod \"console-operator-68b757fb67-pth6b\" (UID: \"44d046d2-51c5-4769-a665-1805b2c28340\") " pod="openshift-console-operator/console-operator-68b757fb67-pth6b" Jul 06 16:38:54.657200 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.655071 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/44d046d2-51c5-4769-a665-1805b2c28340-trusted-ca\") pod \"console-operator-68b757fb67-pth6b\" (UID: \"44d046d2-51c5-4769-a665-1805b2c28340\") " pod="openshift-console-operator/console-operator-68b757fb67-pth6b" Jul 06 16:38:54.659726 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.659698 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/44d046d2-51c5-4769-a665-1805b2c28340-serving-cert\") pod \"console-operator-68b757fb67-pth6b\" (UID: \"44d046d2-51c5-4769-a665-1805b2c28340\") " pod="openshift-console-operator/console-operator-68b757fb67-pth6b" Jul 06 16:38:54.660316 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.660253 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2372b154-c850-4003-ba8a-3534dd9c8cde-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-4ggmp\" (UID: \"2372b154-c850-4003-ba8a-3534dd9c8cde\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4ggmp" Jul 06 16:38:54.687271 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.687244 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jp9x2\" (UniqueName: \"kubernetes.io/projected/2372b154-c850-4003-ba8a-3534dd9c8cde-kube-api-access-jp9x2\") pod \"service-ca-operator-5d46d9ffd4-4ggmp\" (UID: \"2372b154-c850-4003-ba8a-3534dd9c8cde\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4ggmp" Jul 06 16:38:54.687989 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.687957 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9snlc\" (UniqueName: \"kubernetes.io/projected/44d046d2-51c5-4769-a665-1805b2c28340-kube-api-access-9snlc\") pod \"console-operator-68b757fb67-pth6b\" (UID: \"44d046d2-51c5-4769-a665-1805b2c28340\") " pod="openshift-console-operator/console-operator-68b757fb67-pth6b" Jul 06 16:38:54.712578 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.712506 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-7dtv6"] Jul 06 16:38:54.716003 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:54.715981 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod861b3db1_e952_457e_abdd_d36829821c19.slice/crio-5211c85020d458bd0f8970e89cbda7276f8c5925ca0e8750fc784772aefc93a4 WatchSource:0}: Error finding container 5211c85020d458bd0f8970e89cbda7276f8c5925ca0e8750fc784772aefc93a4: Status 404 returned error can't find the container with id 5211c85020d458bd0f8970e89cbda7276f8c5925ca0e8750fc784772aefc93a4 Jul 06 16:38:54.771200 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.771174 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-pth6b" Jul 06 16:38:54.795313 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.795289 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4ggmp" Jul 06 16:38:54.855574 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.855523 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/36d6010c-f9cd-4fea-9e77-3cf687d3616b-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ht4xk\" (UID: \"36d6010c-f9cd-4fea-9e77-3cf687d3616b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht4xk" Jul 06 16:38:54.856055 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:54.855785 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 16:38:54.856055 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:54.855871 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36d6010c-f9cd-4fea-9e77-3cf687d3616b-samples-operator-tls podName:36d6010c-f9cd-4fea-9e77-3cf687d3616b nodeName:}" failed. No retries permitted until 2026-07-06 16:38:56.855849253 +0000 UTC m=+51.033708362 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/36d6010c-f9cd-4fea-9e77-3cf687d3616b-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-ht4xk" (UID: "36d6010c-f9cd-4fea-9e77-3cf687d3616b") : secret "samples-operator-tls" not found Jul 06 16:38:54.963358 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.963287 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-pth6b"] Jul 06 16:38:54.966497 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:54.966463 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44d046d2_51c5_4769_a665_1805b2c28340.slice/crio-027f2bd540296f4797740712570723e68b36663f8dfec312813f764b31269be4 WatchSource:0}: Error finding container 027f2bd540296f4797740712570723e68b36663f8dfec312813f764b31269be4: Status 404 returned error can't find the container with id 027f2bd540296f4797740712570723e68b36663f8dfec312813f764b31269be4 Jul 06 16:38:54.982400 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:54.982377 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4ggmp"] Jul 06 16:38:54.984997 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:38:54.984972 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2372b154_c850_4003_ba8a_3534dd9c8cde.slice/crio-a637646b5f8ea540e39ec388bc22bb28bcc24d9fa414dbc03c5d502fa9690abe WatchSource:0}: Error finding container a637646b5f8ea540e39ec388bc22bb28bcc24d9fa414dbc03c5d502fa9690abe: Status 404 returned error can't find the container with id a637646b5f8ea540e39ec388bc22bb28bcc24d9fa414dbc03c5d502fa9690abe Jul 06 16:38:55.479464 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:55.479432 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-7dtv6" event={"ID":"861b3db1-e952-457e-abdd-d36829821c19","Type":"ContainerStarted","Data":"5211c85020d458bd0f8970e89cbda7276f8c5925ca0e8750fc784772aefc93a4"} Jul 06 16:38:55.480445 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:55.480423 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4ggmp" event={"ID":"2372b154-c850-4003-ba8a-3534dd9c8cde","Type":"ContainerStarted","Data":"a637646b5f8ea540e39ec388bc22bb28bcc24d9fa414dbc03c5d502fa9690abe"} Jul 06 16:38:55.481341 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:55.481322 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-pth6b" event={"ID":"44d046d2-51c5-4769-a665-1805b2c28340","Type":"ContainerStarted","Data":"027f2bd540296f4797740712570723e68b36663f8dfec312813f764b31269be4"} Jul 06 16:38:56.485267 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:56.485232 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mct68" event={"ID":"5d998ace-6b9c-435e-9360-b0776df34e66","Type":"ContainerStarted","Data":"81fe997fe33902fc5f198dbb5bc18f8a6647043a5acc6e6f85563559b722435f"} Jul 06 16:38:56.520378 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:56.520309 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-mct68" podStartSLOduration=1.492408033 podStartE2EDuration="3.520289411s" totalImagesPullingTime="2.027881378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:38:56.519847873 +0000 UTC m=+50.697706987" watchObservedRunningTime="2026-07-06 16:38:56.520289411 +0000 UTC m=+50.698148524" Jul 06 16:38:56.875855 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:56.875818 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/36d6010c-f9cd-4fea-9e77-3cf687d3616b-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ht4xk\" (UID: \"36d6010c-f9cd-4fea-9e77-3cf687d3616b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht4xk" Jul 06 16:38:56.876022 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:56.875956 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 16:38:56.876067 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:56.876036 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36d6010c-f9cd-4fea-9e77-3cf687d3616b-samples-operator-tls podName:36d6010c-f9cd-4fea-9e77-3cf687d3616b nodeName:}" failed. No retries permitted until 2026-07-06 16:39:00.876014386 +0000 UTC m=+55.053873499 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/36d6010c-f9cd-4fea-9e77-3cf687d3616b-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-ht4xk" (UID: "36d6010c-f9cd-4fea-9e77-3cf687d3616b") : secret "samples-operator-tls" not found Jul 06 16:38:58.590140 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:58.590104 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:58.590675 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:38:58.590222 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:38:58.590675 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:58.590238 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 16:38:58.590675 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:58.590301 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-kube-rbac-proxy-cm podName:6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e nodeName:}" failed. No retries permitted until 2026-07-06 16:39:06.590286336 +0000 UTC m=+60.768145425 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-ps5hn" (UID: "6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e") : configmap "kube-rbac-proxy" not found Jul 06 16:38:58.590675 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:58.590393 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 16:38:58.590675 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:38:58.590457 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-insights-runtime-extractor-tls podName:6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e nodeName:}" failed. No retries permitted until 2026-07-06 16:39:06.590441135 +0000 UTC m=+60.768300228 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ps5hn" (UID: "6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e") : secret "insights-runtime-extractor-tls" not found Jul 06 16:39:00.497384 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:00.497343 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-7dtv6" event={"ID":"861b3db1-e952-457e-abdd-d36829821c19","Type":"ContainerStarted","Data":"078d30bdc84414bfc3ef69cfb944003b27d4c29cdc5bbde548ddcd5b27c6dc19"} Jul 06 16:39:00.498771 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:00.498749 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4ggmp" event={"ID":"2372b154-c850-4003-ba8a-3534dd9c8cde","Type":"ContainerStarted","Data":"6d906295715f1cd2c8bcc81a192acf45829bf8f2e834cf4ecf64558704a55a1c"} Jul 06 16:39:00.500226 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:00.500208 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/0.log" Jul 06 16:39:00.500305 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:00.500239 2580 generic.go:356] "Generic (PLEG): container finished" podID="44d046d2-51c5-4769-a665-1805b2c28340" containerID="e2602fc215eb57abe201ead96a803cb6a6a94e43e987e9f4ebd30c668c501c08" exitCode=255 Jul 06 16:39:00.500305 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:00.500272 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-pth6b" event={"ID":"44d046d2-51c5-4769-a665-1805b2c28340","Type":"ContainerDied","Data":"e2602fc215eb57abe201ead96a803cb6a6a94e43e987e9f4ebd30c668c501c08"} Jul 06 16:39:00.500527 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:00.500510 2580 scope.go:117] "RemoveContainer" containerID="e2602fc215eb57abe201ead96a803cb6a6a94e43e987e9f4ebd30c668c501c08" Jul 06 16:39:00.567835 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:00.567674 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-7dtv6" podStartSLOduration=1.8316742270000002 podStartE2EDuration="6.567651728s" totalImagesPullingTime="4.735977501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:38:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:39:00.529630259 +0000 UTC m=+54.707489371" watchObservedRunningTime="2026-07-06 16:39:00.567651728 +0000 UTC m=+54.745510843" Jul 06 16:39:00.620773 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:00.620729 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4ggmp" podStartSLOduration=2.143844155 podStartE2EDuration="6.620712994s" totalImagesPullingTime="4.476868839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:38:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:39:00.619089637 +0000 UTC m=+54.796948749" watchObservedRunningTime="2026-07-06 16:39:00.620712994 +0000 UTC m=+54.798572105" Jul 06 16:39:00.912486 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:00.912434 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/36d6010c-f9cd-4fea-9e77-3cf687d3616b-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ht4xk\" (UID: \"36d6010c-f9cd-4fea-9e77-3cf687d3616b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht4xk" Jul 06 16:39:00.912644 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:00.912596 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 16:39:00.912693 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:00.912676 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36d6010c-f9cd-4fea-9e77-3cf687d3616b-samples-operator-tls podName:36d6010c-f9cd-4fea-9e77-3cf687d3616b nodeName:}" failed. No retries permitted until 2026-07-06 16:39:08.912660188 +0000 UTC m=+63.090519277 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/36d6010c-f9cd-4fea-9e77-3cf687d3616b-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-ht4xk" (UID: "36d6010c-f9cd-4fea-9e77-3cf687d3616b") : secret "samples-operator-tls" not found Jul 06 16:39:01.114554 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:01.114516 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:39:01.114713 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:01.114667 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 16:39:01.114713 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:01.114687 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c58c78ccb-5sns6: secret "image-registry-tls" not found Jul 06 16:39:01.114782 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:01.114740 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 16:39:01.114782 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:01.114763 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls podName:18464d2a-4b04-486f-bfc2-ac4d21f31578 nodeName:}" failed. No retries permitted until 2026-07-06 16:39:17.114747855 +0000 UTC m=+71.292606945 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls") pod "image-registry-7c58c78ccb-5sns6" (UID: "18464d2a-4b04-486f-bfc2-ac4d21f31578") : secret "image-registry-tls" not found Jul 06 16:39:01.114853 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:01.114793 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls podName:59df778e-ece6-45e2-a0df-551df597ee79 nodeName:}" failed. No retries permitted until 2026-07-06 16:39:17.114769411 +0000 UTC m=+71.292628526 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-x247s" (UID: "59df778e-ece6-45e2-a0df-551df597ee79") : secret "cluster-monitoring-operator-tls" not found Jul 06 16:39:01.114853 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:01.114669 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x247s\" (UID: \"59df778e-ece6-45e2-a0df-551df597ee79\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s" Jul 06 16:39:01.215484 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:01.215391 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:39:01.215484 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:01.215453 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0604265-3597-4501-9b25-8765c41d7f49-metrics-tls\") pod \"dns-default-mrznn\" (UID: \"d0604265-3597-4501-9b25-8765c41d7f49\") " pod="openshift-dns/dns-default-mrznn" Jul 06 16:39:01.215680 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:01.215525 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-metrics-certs\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:39:01.215680 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:01.215559 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 16:39:01.215680 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:01.215574 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle podName:f7221e3e-0a2d-4d91-b0f1-1b303255b302 nodeName:}" failed. No retries permitted until 2026-07-06 16:39:17.215554808 +0000 UTC m=+71.393413898 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle") pod "router-default-87b54f4c7-4lpwt" (UID: "f7221e3e-0a2d-4d91-b0f1-1b303255b302") : configmap references non-existent config key: service-ca.crt Jul 06 16:39:01.215680 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:01.215601 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0604265-3597-4501-9b25-8765c41d7f49-metrics-tls podName:d0604265-3597-4501-9b25-8765c41d7f49 nodeName:}" failed. No retries permitted until 2026-07-06 16:39:17.215589689 +0000 UTC m=+71.393448802 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d0604265-3597-4501-9b25-8765c41d7f49-metrics-tls") pod "dns-default-mrznn" (UID: "d0604265-3597-4501-9b25-8765c41d7f49") : secret "dns-default-metrics-tls" not found Jul 06 16:39:01.215680 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:01.215611 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 16:39:01.215680 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:01.215659 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-metrics-certs podName:f7221e3e-0a2d-4d91-b0f1-1b303255b302 nodeName:}" failed. No retries permitted until 2026-07-06 16:39:17.215648442 +0000 UTC m=+71.393507533 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-metrics-certs") pod "router-default-87b54f4c7-4lpwt" (UID: "f7221e3e-0a2d-4d91-b0f1-1b303255b302") : secret "router-metrics-certs-default" not found Jul 06 16:39:01.504727 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:01.504652 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 16:39:01.505121 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:01.505107 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/0.log" Jul 06 16:39:01.505179 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:01.505139 2580 generic.go:356] "Generic (PLEG): container finished" podID="44d046d2-51c5-4769-a665-1805b2c28340" containerID="55f92e01810439b43c3a1b17806145caf4818e37c0977d9735906a74c2f622d6" exitCode=255 Jul 06 16:39:01.505253 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:01.505229 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-pth6b" event={"ID":"44d046d2-51c5-4769-a665-1805b2c28340","Type":"ContainerDied","Data":"55f92e01810439b43c3a1b17806145caf4818e37c0977d9735906a74c2f622d6"} Jul 06 16:39:01.505307 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:01.505283 2580 scope.go:117] "RemoveContainer" containerID="e2602fc215eb57abe201ead96a803cb6a6a94e43e987e9f4ebd30c668c501c08" Jul 06 16:39:01.505615 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:01.505519 2580 scope.go:117] "RemoveContainer" containerID="55f92e01810439b43c3a1b17806145caf4818e37c0977d9735906a74c2f622d6" Jul 06 16:39:01.505744 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:01.505721 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-pth6b_openshift-console-operator(44d046d2-51c5-4769-a665-1805b2c28340)\"" pod="openshift-console-operator/console-operator-68b757fb67-pth6b" podUID="44d046d2-51c5-4769-a665-1805b2c28340" Jul 06 16:39:02.509423 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:02.509395 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 16:39:02.509937 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:02.509796 2580 scope.go:117] "RemoveContainer" containerID="55f92e01810439b43c3a1b17806145caf4818e37c0977d9735906a74c2f622d6" Jul 06 16:39:02.510029 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:02.510002 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-pth6b_openshift-console-operator(44d046d2-51c5-4769-a665-1805b2c28340)\"" pod="openshift-console-operator/console-operator-68b757fb67-pth6b" podUID="44d046d2-51c5-4769-a665-1805b2c28340" Jul 06 16:39:04.771782 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:04.771742 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-pth6b" Jul 06 16:39:04.772256 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:04.771816 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-pth6b" Jul 06 16:39:04.772256 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:04.772226 2580 scope.go:117] "RemoveContainer" containerID="55f92e01810439b43c3a1b17806145caf4818e37c0977d9735906a74c2f622d6" Jul 06 16:39:04.772426 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:04.772402 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-pth6b_openshift-console-operator(44d046d2-51c5-4769-a665-1805b2c28340)\"" pod="openshift-console-operator/console-operator-68b757fb67-pth6b" podUID="44d046d2-51c5-4769-a665-1805b2c28340" Jul 06 16:39:06.664648 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:06.664605 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:39:06.665027 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:06.664743 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:39:06.665027 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:06.664806 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 16:39:06.665027 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:06.664867 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-insights-runtime-extractor-tls podName:6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e nodeName:}" failed. No retries permitted until 2026-07-06 16:39:22.664853183 +0000 UTC m=+76.842712276 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-ps5hn" (UID: "6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e") : secret "insights-runtime-extractor-tls" not found Jul 06 16:39:06.665273 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:06.665253 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:39:08.986443 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:08.986403 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/36d6010c-f9cd-4fea-9e77-3cf687d3616b-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ht4xk\" (UID: \"36d6010c-f9cd-4fea-9e77-3cf687d3616b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht4xk" Jul 06 16:39:08.986942 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:08.986565 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 16:39:08.986942 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:08.986632 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36d6010c-f9cd-4fea-9e77-3cf687d3616b-samples-operator-tls podName:36d6010c-f9cd-4fea-9e77-3cf687d3616b nodeName:}" failed. No retries permitted until 2026-07-06 16:39:24.986615642 +0000 UTC m=+79.164474732 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/36d6010c-f9cd-4fea-9e77-3cf687d3616b-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-ht4xk" (UID: "36d6010c-f9cd-4fea-9e77-3cf687d3616b") : secret "samples-operator-tls" not found Jul 06 16:39:09.438123 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:09.438100 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jrb4n" Jul 06 16:39:12.014005 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:12.013968 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs\") pod \"network-metrics-daemon-5m692\" (UID: \"3f867a4a-dbf4-46a7-a79b-c30d916fa0f8\") " pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:39:12.023187 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:12.023161 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 16:39:12.025052 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:12.025036 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 16:39:12.025110 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:12.025094 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs podName:3f867a4a-dbf4-46a7-a79b-c30d916fa0f8 nodeName:}" failed. No retries permitted until 2026-07-06 16:40:16.025075825 +0000 UTC m=+130.202934914 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs") pod "network-metrics-daemon-5m692" (UID: "3f867a4a-dbf4-46a7-a79b-c30d916fa0f8") : secret "metrics-daemon-secret" not found Jul 06 16:39:12.114907 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:12.114876 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret\") pod \"global-pull-secret-syncer-2m87r\" (UID: \"0f5cbd3c-3a54-4441-bea2-2643a5912979\") " pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:39:12.118669 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:12.118652 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 16:39:12.127810 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:12.127780 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0f5cbd3c-3a54-4441-bea2-2643a5912979-original-pull-secret\") pod \"global-pull-secret-syncer-2m87r\" (UID: \"0f5cbd3c-3a54-4441-bea2-2643a5912979\") " pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:39:12.215756 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:12.215730 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9bd9\" (UniqueName: \"kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9\") pod \"network-check-target-xzfns\" (UID: \"52ebf132-7501-49f4-8615-28020de19a6f\") " pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:39:12.218158 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:12.218141 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 16:39:12.228695 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:12.228675 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 16:39:12.239716 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:12.239699 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9bd9\" (UniqueName: \"kubernetes.io/projected/52ebf132-7501-49f4-8615-28020de19a6f-kube-api-access-p9bd9\") pod \"network-check-target-xzfns\" (UID: \"52ebf132-7501-49f4-8615-28020de19a6f\") " pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:39:12.376428 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:12.376401 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2m87r" Jul 06 16:39:12.388661 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:12.388640 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q4r89\"" Jul 06 16:39:12.393783 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:12.393762 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:39:12.532147 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:12.532117 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2m87r"] Jul 06 16:39:12.535141 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:39:12.535111 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f5cbd3c_3a54_4441_bea2_2643a5912979.slice/crio-f722e49da9734198ff8811fdc9522d6dcc096143a1b6ffe9959cf5d8bdfdee94 WatchSource:0}: Error finding container f722e49da9734198ff8811fdc9522d6dcc096143a1b6ffe9959cf5d8bdfdee94: Status 404 returned error can't find the container with id f722e49da9734198ff8811fdc9522d6dcc096143a1b6ffe9959cf5d8bdfdee94 Jul 06 16:39:12.539596 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:12.539556 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2m87r" event={"ID":"0f5cbd3c-3a54-4441-bea2-2643a5912979","Type":"ContainerStarted","Data":"f722e49da9734198ff8811fdc9522d6dcc096143a1b6ffe9959cf5d8bdfdee94"} Jul 06 16:39:12.564732 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:12.564712 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xzfns"] Jul 06 16:39:12.566678 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:39:12.566662 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52ebf132_7501_49f4_8615_28020de19a6f.slice/crio-6d193b8dc6ecb1c29ed18ccc92fab96523cec1d018b261ec81e611ccf2dca3fe WatchSource:0}: Error finding container 6d193b8dc6ecb1c29ed18ccc92fab96523cec1d018b261ec81e611ccf2dca3fe: Status 404 returned error can't find the container with id 6d193b8dc6ecb1c29ed18ccc92fab96523cec1d018b261ec81e611ccf2dca3fe Jul 06 16:39:13.543763 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:13.543730 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xzfns" event={"ID":"52ebf132-7501-49f4-8615-28020de19a6f","Type":"ContainerStarted","Data":"6d193b8dc6ecb1c29ed18ccc92fab96523cec1d018b261ec81e611ccf2dca3fe"} Jul 06 16:39:15.258975 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:15.258944 2580 scope.go:117] "RemoveContainer" containerID="55f92e01810439b43c3a1b17806145caf4818e37c0977d9735906a74c2f622d6" Jul 06 16:39:16.557127 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:16.557089 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xzfns" event={"ID":"52ebf132-7501-49f4-8615-28020de19a6f","Type":"ContainerStarted","Data":"f0ed4a3c5e98abb990f1d991b339dc677cbfbcf261cd1aaebd12c1cbf629a916"} Jul 06 16:39:16.557586 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:16.557180 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:39:16.559120 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:16.559097 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 16:39:16.559268 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:16.559155 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-pth6b" event={"ID":"44d046d2-51c5-4769-a665-1805b2c28340","Type":"ContainerStarted","Data":"ebb2620ebabde4f7eb22e47a6fbdee9c37f73bd0b0a8c0e2435edd0d63446252"} Jul 06 16:39:16.559459 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:16.559432 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-pth6b" Jul 06 16:39:16.564285 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:16.564265 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-pth6b" Jul 06 16:39:16.592543 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:16.592502 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xzfns" podStartSLOduration=67.384144204 podStartE2EDuration="1m10.592490472s" totalImagesPullingTime="3.208346268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:39:16.592321986 +0000 UTC m=+70.770181259" watchObservedRunningTime="2026-07-06 16:39:16.592490472 +0000 UTC m=+70.770349578" Jul 06 16:39:16.647266 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:16.647218 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-pth6b" podStartSLOduration=18.161205061 podStartE2EDuration="22.647203707s" totalImagesPullingTime="4.485998646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:38:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:39:16.644409539 +0000 UTC m=+70.822268651" watchObservedRunningTime="2026-07-06 16:39:16.647203707 +0000 UTC m=+70.825062819" Jul 06 16:39:17.160825 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:17.160788 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x247s\" (UID: \"59df778e-ece6-45e2-a0df-551df597ee79\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s" Jul 06 16:39:17.161023 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:17.160847 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:39:17.161023 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:17.160937 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 16:39:17.161023 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:17.161014 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls podName:59df778e-ece6-45e2-a0df-551df597ee79 nodeName:}" failed. No retries permitted until 2026-07-06 16:39:49.160994367 +0000 UTC m=+103.338853458 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-x247s" (UID: "59df778e-ece6-45e2-a0df-551df597ee79") : secret "cluster-monitoring-operator-tls" not found Jul 06 16:39:17.163236 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:17.163212 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls\") pod \"image-registry-7c58c78ccb-5sns6\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:39:17.261688 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:17.261658 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0604265-3597-4501-9b25-8765c41d7f49-metrics-tls\") pod \"dns-default-mrznn\" (UID: \"d0604265-3597-4501-9b25-8765c41d7f49\") " pod="openshift-dns/dns-default-mrznn" Jul 06 16:39:17.261849 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:17.261827 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-metrics-certs\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:39:17.261950 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:17.261934 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:39:17.262076 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:17.262061 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle podName:f7221e3e-0a2d-4d91-b0f1-1b303255b302 nodeName:}" failed. No retries permitted until 2026-07-06 16:39:49.262043073 +0000 UTC m=+103.439902163 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle") pod "router-default-87b54f4c7-4lpwt" (UID: "f7221e3e-0a2d-4d91-b0f1-1b303255b302") : configmap references non-existent config key: service-ca.crt Jul 06 16:39:17.263988 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:17.263963 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0604265-3597-4501-9b25-8765c41d7f49-metrics-tls\") pod \"dns-default-mrznn\" (UID: \"d0604265-3597-4501-9b25-8765c41d7f49\") " pod="openshift-dns/dns-default-mrznn" Jul 06 16:39:17.264094 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:17.264015 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7221e3e-0a2d-4d91-b0f1-1b303255b302-metrics-certs\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:39:17.361934 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:17.361909 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8rjds\"" Jul 06 16:39:17.369939 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:17.369922 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:39:17.437507 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:17.437434 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-db7nl\"" Jul 06 16:39:17.446247 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:17.446225 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-mrznn" Jul 06 16:39:17.526238 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:17.526110 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c58c78ccb-5sns6"] Jul 06 16:39:17.528919 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:39:17.528849 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18464d2a_4b04_486f_bfc2_ac4d21f31578.slice/crio-dae2b0d17603bee1437f3ecd022f77c4b444f7d0a9e01c3a2a3628f537f38290 WatchSource:0}: Error finding container dae2b0d17603bee1437f3ecd022f77c4b444f7d0a9e01c3a2a3628f537f38290: Status 404 returned error can't find the container with id dae2b0d17603bee1437f3ecd022f77c4b444f7d0a9e01c3a2a3628f537f38290 Jul 06 16:39:17.564917 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:17.564884 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" event={"ID":"18464d2a-4b04-486f-bfc2-ac4d21f31578","Type":"ContainerStarted","Data":"dae2b0d17603bee1437f3ecd022f77c4b444f7d0a9e01c3a2a3628f537f38290"} Jul 06 16:39:17.615279 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:17.615253 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-mrznn"] Jul 06 16:39:17.622798 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:39:17.622775 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0604265_3597_4501_9b25_8765c41d7f49.slice/crio-1174f984017f234b49d5911439eb934d20a741dd7f68bfcaa18447af41acaf0a WatchSource:0}: Error finding container 1174f984017f234b49d5911439eb934d20a741dd7f68bfcaa18447af41acaf0a: Status 404 returned error can't find the container with id 1174f984017f234b49d5911439eb934d20a741dd7f68bfcaa18447af41acaf0a Jul 06 16:39:18.569832 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:18.569797 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" event={"ID":"18464d2a-4b04-486f-bfc2-ac4d21f31578","Type":"ContainerStarted","Data":"af1502ac2982ba50cf5661e4bf082c68aabafee86038d66c6b7302e3b46b943e"} Jul 06 16:39:18.570292 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:18.569881 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:39:18.571170 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:18.571141 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mrznn" event={"ID":"d0604265-3597-4501-9b25-8765c41d7f49","Type":"ContainerStarted","Data":"1174f984017f234b49d5911439eb934d20a741dd7f68bfcaa18447af41acaf0a"} Jul 06 16:39:18.609041 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:18.608984 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" podStartSLOduration=71.608964793 podStartE2EDuration="1m11.608964793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:38:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:39:18.608184425 +0000 UTC m=+72.786043537" watchObservedRunningTime="2026-07-06 16:39:18.608964793 +0000 UTC m=+72.786823906" Jul 06 16:39:19.581226 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:19.581144 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2m87r" event={"ID":"0f5cbd3c-3a54-4441-bea2-2643a5912979","Type":"ContainerStarted","Data":"602cd954f761db7fdb33e71b0c283c65ff2d622f8c1439da122dbb7ef0717379"} Jul 06 16:39:19.637109 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:19.637062 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2m87r" podStartSLOduration=65.908841285 podStartE2EDuration="1m12.637045566s" totalImagesPullingTime="6.728204281s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:38:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:39:19.636621932 +0000 UTC m=+73.814481055" watchObservedRunningTime="2026-07-06 16:39:19.637045566 +0000 UTC m=+73.814904680" Jul 06 16:39:20.585300 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:20.585272 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mrznn" event={"ID":"d0604265-3597-4501-9b25-8765c41d7f49","Type":"ContainerStarted","Data":"2d761c671d9f89edad4c24bfd2fdb7d2a21ff116f2cab1edd1c371121b2ab7f6"} Jul 06 16:39:21.590365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:21.590326 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-mrznn" event={"ID":"d0604265-3597-4501-9b25-8765c41d7f49","Type":"ContainerStarted","Data":"5afdf31ddaa0d124ba9b7022173d5e68a5e756932c5b5d43f588aa160c493356"} Jul 06 16:39:21.590814 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:21.590481 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-mrznn" Jul 06 16:39:21.633450 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:21.633398 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-mrznn" podStartSLOduration=33.845179334 podStartE2EDuration="36.633383476s" totalImagesPullingTime="2.788204142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:38:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:39:21.632637278 +0000 UTC m=+75.810496390" watchObservedRunningTime="2026-07-06 16:39:21.633383476 +0000 UTC m=+75.811242588" Jul 06 16:39:22.709116 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:22.709073 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:39:22.711389 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:22.711367 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-ps5hn\" (UID: \"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e\") " pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:39:22.935055 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:22.935031 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nl5jr\"" Jul 06 16:39:22.942535 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:22.942515 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-ps5hn" Jul 06 16:39:23.080210 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:23.080177 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-ps5hn"] Jul 06 16:39:23.083296 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:39:23.083263 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6046edef_fd1a_4e09_a7f5_a6b9a3f70c8e.slice/crio-8ca41a57f715c8786c5cbf6ea3aa552721768c1390ba5e3e73641072a38d293d WatchSource:0}: Error finding container 8ca41a57f715c8786c5cbf6ea3aa552721768c1390ba5e3e73641072a38d293d: Status 404 returned error can't find the container with id 8ca41a57f715c8786c5cbf6ea3aa552721768c1390ba5e3e73641072a38d293d Jul 06 16:39:23.600451 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:23.600410 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ps5hn" event={"ID":"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e","Type":"ContainerStarted","Data":"8ca41a57f715c8786c5cbf6ea3aa552721768c1390ba5e3e73641072a38d293d"} Jul 06 16:39:25.034610 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:25.034576 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/36d6010c-f9cd-4fea-9e77-3cf687d3616b-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ht4xk\" (UID: \"36d6010c-f9cd-4fea-9e77-3cf687d3616b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht4xk" Jul 06 16:39:25.037193 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:25.037161 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/36d6010c-f9cd-4fea-9e77-3cf687d3616b-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-ht4xk\" (UID: \"36d6010c-f9cd-4fea-9e77-3cf687d3616b\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht4xk" Jul 06 16:39:25.357090 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:25.357057 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6zmpk\"" Jul 06 16:39:25.359081 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:25.359056 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht4xk" Jul 06 16:39:26.873812 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:26.873787 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht4xk"] Jul 06 16:39:27.618312 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:27.618274 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ps5hn" event={"ID":"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e","Type":"ContainerStarted","Data":"6d81713799b2f7d65a70433beeb29e099af98554233db69b6b628351e76d30d8"} Jul 06 16:39:27.619532 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:27.619501 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht4xk" event={"ID":"36d6010c-f9cd-4fea-9e77-3cf687d3616b","Type":"ContainerStarted","Data":"119e3d6390eb7a3fd9cb3815eed585c706baf125b3bf87d411f43701f6fb44ca"} Jul 06 16:39:29.628522 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:29.628459 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ps5hn" event={"ID":"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e","Type":"ContainerStarted","Data":"31eee1c62d498d98b6fa308c86cdcaa649f798caff284df053a2ae93b2757552"} Jul 06 16:39:29.630281 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:29.630255 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht4xk" event={"ID":"36d6010c-f9cd-4fea-9e77-3cf687d3616b","Type":"ContainerStarted","Data":"77fdf9afbbd55807f622e9407cf7027b3dae56d389b7ce2b8246e90acd1dfb89"} Jul 06 16:39:29.630366 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:29.630298 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht4xk" event={"ID":"36d6010c-f9cd-4fea-9e77-3cf687d3616b","Type":"ContainerStarted","Data":"18708e633f5ce91db49c44a46c7f507c5a7cf3fd12fc55e85d819e1486b394d1"} Jul 06 16:39:29.654181 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:29.654139 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-ht4xk" podStartSLOduration=34.256386356 podStartE2EDuration="36.654126272s" totalImagesPullingTime="2.397739916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:39:29.654000263 +0000 UTC m=+83.831859376" watchObservedRunningTime="2026-07-06 16:39:29.654126272 +0000 UTC m=+83.831985362" Jul 06 16:39:31.595909 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:31.595875 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-mrznn" Jul 06 16:39:32.643548 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:32.643511 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-ps5hn" event={"ID":"6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e","Type":"ContainerStarted","Data":"a6ec8e8975cac1459b2756ecda59129200b02b7d4618b46902c4dcdc53e60e76"} Jul 06 16:39:32.680732 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:32.680684 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-ps5hn" podStartSLOduration=33.770082168 podStartE2EDuration="42.680667647s" totalImagesPullingTime="8.910585479s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 16:38:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:39:32.680323056 +0000 UTC m=+86.858182168" watchObservedRunningTime="2026-07-06 16:39:32.680667647 +0000 UTC m=+86.858526759" Jul 06 16:39:37.374819 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:37.374784 2580 patch_prober.go:28] interesting pod/image-registry-7c58c78ccb-5sns6 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 06 16:39:37.375270 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:37.374844 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" podUID="18464d2a-4b04-486f-bfc2-ac4d21f31578" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 16:39:39.586051 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:39.586024 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:39:47.125056 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:47.125019 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7c58c78ccb-5sns6"] Jul 06 16:39:47.568111 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:47.568031 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xzfns" Jul 06 16:39:49.220352 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:49.220313 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x247s\" (UID: \"59df778e-ece6-45e2-a0df-551df597ee79\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s" Jul 06 16:39:49.222762 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:49.222743 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/59df778e-ece6-45e2-a0df-551df597ee79-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-x247s\" (UID: \"59df778e-ece6-45e2-a0df-551df597ee79\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s" Jul 06 16:39:49.320936 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:49.320898 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:39:49.321496 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:49.321459 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f7221e3e-0a2d-4d91-b0f1-1b303255b302-service-ca-bundle\") pod \"router-default-87b54f4c7-4lpwt\" (UID: \"f7221e3e-0a2d-4d91-b0f1-1b303255b302\") " pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:39:49.475254 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:49.475184 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-dj8sn\"" Jul 06 16:39:49.483602 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:49.483585 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s" Jul 06 16:39:49.517877 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:49.517853 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-szx98\"" Jul 06 16:39:49.526019 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:49.525998 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:39:49.628824 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:49.628793 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s"] Jul 06 16:39:49.634247 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:39:49.634210 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59df778e_ece6_45e2_a0df_551df597ee79.slice/crio-f271f452bdfb6a54412f4b5a4c1d850358a5edbb838624b6042408f33dae34fc WatchSource:0}: Error finding container f271f452bdfb6a54412f4b5a4c1d850358a5edbb838624b6042408f33dae34fc: Status 404 returned error can't find the container with id f271f452bdfb6a54412f4b5a4c1d850358a5edbb838624b6042408f33dae34fc Jul 06 16:39:49.670337 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:49.670306 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-87b54f4c7-4lpwt"] Jul 06 16:39:49.675010 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:39:49.674980 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7221e3e_0a2d_4d91_b0f1_1b303255b302.slice/crio-14c35712b52f2dc3bcd4beb328917233d8f7423168101fcab0a059f058c8a423 WatchSource:0}: Error finding container 14c35712b52f2dc3bcd4beb328917233d8f7423168101fcab0a059f058c8a423: Status 404 returned error can't find the container with id 14c35712b52f2dc3bcd4beb328917233d8f7423168101fcab0a059f058c8a423 Jul 06 16:39:49.698776 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:49.698752 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-87b54f4c7-4lpwt" event={"ID":"f7221e3e-0a2d-4d91-b0f1-1b303255b302","Type":"ContainerStarted","Data":"14c35712b52f2dc3bcd4beb328917233d8f7423168101fcab0a059f058c8a423"} Jul 06 16:39:49.699808 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:49.699784 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s" event={"ID":"59df778e-ece6-45e2-a0df-551df597ee79","Type":"ContainerStarted","Data":"f271f452bdfb6a54412f4b5a4c1d850358a5edbb838624b6042408f33dae34fc"} Jul 06 16:39:50.704463 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:50.704427 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-87b54f4c7-4lpwt" event={"ID":"f7221e3e-0a2d-4d91-b0f1-1b303255b302","Type":"ContainerStarted","Data":"24b2d0de8b96b994de5c46d17d08e32b0521cf7f59f41fdadde5ea2c096af05f"} Jul 06 16:39:50.746520 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:50.746451 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-87b54f4c7-4lpwt" podStartSLOduration=66.746436866 podStartE2EDuration="1m6.746436866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:38:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:39:50.746283969 +0000 UTC m=+104.924143080" watchObservedRunningTime="2026-07-06 16:39:50.746436866 +0000 UTC m=+104.924295978" Jul 06 16:39:51.526558 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:51.526520 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:39:51.529311 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:51.529287 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:39:51.708204 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:51.708179 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:39:51.709321 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:51.709298 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-87b54f4c7-4lpwt" Jul 06 16:39:53.371031 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:53.370998 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-42c8w"] Jul 06 16:39:53.383481 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:53.383452 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-42c8w" Jul 06 16:39:53.386261 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:53.386239 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 06 16:39:53.386370 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:53.386333 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-cm2r7\"" Jul 06 16:39:53.391928 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:53.391851 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-42c8w"] Jul 06 16:39:53.458152 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:53.458114 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/eb8b2f4a-3d04-4aa1-beb9-02be75dc54c7-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-42c8w\" (UID: \"eb8b2f4a-3d04-4aa1-beb9-02be75dc54c7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-42c8w" Jul 06 16:39:53.559264 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:53.559231 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/eb8b2f4a-3d04-4aa1-beb9-02be75dc54c7-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-42c8w\" (UID: \"eb8b2f4a-3d04-4aa1-beb9-02be75dc54c7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-42c8w" Jul 06 16:39:53.559434 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:53.559349 2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 06 16:39:53.559434 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:53.559408 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb8b2f4a-3d04-4aa1-beb9-02be75dc54c7-tls-certificates podName:eb8b2f4a-3d04-4aa1-beb9-02be75dc54c7 nodeName:}" failed. No retries permitted until 2026-07-06 16:39:54.059393967 +0000 UTC m=+108.237253058 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/eb8b2f4a-3d04-4aa1-beb9-02be75dc54c7-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-42c8w" (UID: "eb8b2f4a-3d04-4aa1-beb9-02be75dc54c7") : secret "prometheus-operator-admission-webhook-tls" not found Jul 06 16:39:53.717945 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:53.717859 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s" event={"ID":"59df778e-ece6-45e2-a0df-551df597ee79","Type":"ContainerStarted","Data":"34170c5fd345507e9004d4cf7eb9689ea11923750e655f4e3fd0a14fa925e6c9"} Jul 06 16:39:53.746728 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:53.746683 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-x247s" podStartSLOduration=66.70735195500001 podStartE2EDuration="1m9.746667664s" totalImagesPullingTime="3.039315709s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:38:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:39:53.746270536 +0000 UTC m=+107.924129645" watchObservedRunningTime="2026-07-06 16:39:53.746667664 +0000 UTC m=+107.924526775" Jul 06 16:39:54.062963 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:54.062872 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/eb8b2f4a-3d04-4aa1-beb9-02be75dc54c7-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-42c8w\" (UID: \"eb8b2f4a-3d04-4aa1-beb9-02be75dc54c7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-42c8w" Jul 06 16:39:54.063126 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:54.062987 2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 06 16:39:54.063126 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:39:54.063040 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eb8b2f4a-3d04-4aa1-beb9-02be75dc54c7-tls-certificates podName:eb8b2f4a-3d04-4aa1-beb9-02be75dc54c7 nodeName:}" failed. No retries permitted until 2026-07-06 16:39:55.063025395 +0000 UTC m=+109.240884485 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/eb8b2f4a-3d04-4aa1-beb9-02be75dc54c7-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-42c8w" (UID: "eb8b2f4a-3d04-4aa1-beb9-02be75dc54c7") : secret "prometheus-operator-admission-webhook-tls" not found Jul 06 16:39:55.071795 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:55.071756 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/eb8b2f4a-3d04-4aa1-beb9-02be75dc54c7-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-42c8w\" (UID: \"eb8b2f4a-3d04-4aa1-beb9-02be75dc54c7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-42c8w" Jul 06 16:39:55.074140 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:55.074116 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/eb8b2f4a-3d04-4aa1-beb9-02be75dc54c7-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-42c8w\" (UID: \"eb8b2f4a-3d04-4aa1-beb9-02be75dc54c7\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-42c8w" Jul 06 16:39:55.194464 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:55.194429 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-42c8w" Jul 06 16:39:55.335662 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:55.335638 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-42c8w"] Jul 06 16:39:55.342642 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:39:55.342613 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb8b2f4a_3d04_4aa1_beb9_02be75dc54c7.slice/crio-8147769534e86deaf7131b6f10a6d1b43e49a09db8b989b99552d756bec249bb WatchSource:0}: Error finding container 8147769534e86deaf7131b6f10a6d1b43e49a09db8b989b99552d756bec249bb: Status 404 returned error can't find the container with id 8147769534e86deaf7131b6f10a6d1b43e49a09db8b989b99552d756bec249bb Jul 06 16:39:55.725169 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:55.725071 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-42c8w" event={"ID":"eb8b2f4a-3d04-4aa1-beb9-02be75dc54c7","Type":"ContainerStarted","Data":"8147769534e86deaf7131b6f10a6d1b43e49a09db8b989b99552d756bec249bb"} Jul 06 16:39:55.726292 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:55.726271 2580 generic.go:356] "Generic (PLEG): container finished" podID="b6aabdd3-9fd3-46e0-ac1b-9636787ae735" containerID="746c1a9d05da6e4e68a846e91a0b693eade0cf598bc6d08061fb765d89719b42" exitCode=0 Jul 06 16:39:55.726380 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:55.726305 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" event={"ID":"b6aabdd3-9fd3-46e0-ac1b-9636787ae735","Type":"ContainerDied","Data":"746c1a9d05da6e4e68a846e91a0b693eade0cf598bc6d08061fb765d89719b42"} Jul 06 16:39:55.726638 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:55.726625 2580 scope.go:117] "RemoveContainer" containerID="746c1a9d05da6e4e68a846e91a0b693eade0cf598bc6d08061fb765d89719b42" Jul 06 16:39:56.733207 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:56.733171 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-42c8w" event={"ID":"eb8b2f4a-3d04-4aa1-beb9-02be75dc54c7","Type":"ContainerStarted","Data":"e9985a970acc114dcb04f037ee3cbc0882c24cf10a53d5e1560aa95a122d5c03"} Jul 06 16:39:56.733707 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:56.733295 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-42c8w" Jul 06 16:39:56.735083 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:56.735057 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-cdh55" event={"ID":"b6aabdd3-9fd3-46e0-ac1b-9636787ae735","Type":"ContainerStarted","Data":"0452eb2c97f2a3da4b458ebf6ce3cf3c142b5523e128dcdf002a3847bb20f3c1"} Jul 06 16:39:56.738068 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:56.738049 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-42c8w" Jul 06 16:39:56.838568 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:56.838500 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-42c8w" podStartSLOduration=2.895891572 podStartE2EDuration="3.838451018s" totalImagesPullingTime="942.559446ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:39:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:39:56.77151547 +0000 UTC m=+110.949374583" watchObservedRunningTime="2026-07-06 16:39:56.838451018 +0000 UTC m=+111.016310121" Jul 06 16:39:57.708544 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:57.708507 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 16:39:57.772249 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:57.772222 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-64wml"] Jul 06 16:39:57.775103 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:57.775088 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-64wml" Jul 06 16:39:57.781688 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:57.781669 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 06 16:39:57.782627 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:57.782610 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 06 16:39:57.782742 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:57.782724 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-dn2rf\"" Jul 06 16:39:57.782865 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:57.782851 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 16:39:57.802437 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:57.802413 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-64wml"] Jul 06 16:39:57.898565 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:57.898533 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0be6285e-fbac-4d35-8888-431e53289a2f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-64wml\" (UID: \"0be6285e-fbac-4d35-8888-431e53289a2f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-64wml" Jul 06 16:39:57.898716 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:57.898570 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0be6285e-fbac-4d35-8888-431e53289a2f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-64wml\" (UID: \"0be6285e-fbac-4d35-8888-431e53289a2f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-64wml" Jul 06 16:39:57.898716 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:57.898653 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0be6285e-fbac-4d35-8888-431e53289a2f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-64wml\" (UID: \"0be6285e-fbac-4d35-8888-431e53289a2f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-64wml" Jul 06 16:39:57.898716 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:57.898675 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jg8b\" (UniqueName: \"kubernetes.io/projected/0be6285e-fbac-4d35-8888-431e53289a2f-kube-api-access-4jg8b\") pod \"prometheus-operator-567f7b8b7c-64wml\" (UID: \"0be6285e-fbac-4d35-8888-431e53289a2f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-64wml" Jul 06 16:39:57.907737 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:57.907715 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/2.log" Jul 06 16:39:57.999918 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:57.999837 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0be6285e-fbac-4d35-8888-431e53289a2f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-64wml\" (UID: \"0be6285e-fbac-4d35-8888-431e53289a2f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-64wml" Jul 06 16:39:57.999918 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:57.999877 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jg8b\" (UniqueName: \"kubernetes.io/projected/0be6285e-fbac-4d35-8888-431e53289a2f-kube-api-access-4jg8b\") pod \"prometheus-operator-567f7b8b7c-64wml\" (UID: \"0be6285e-fbac-4d35-8888-431e53289a2f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-64wml" Jul 06 16:39:58.000129 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:57.999923 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0be6285e-fbac-4d35-8888-431e53289a2f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-64wml\" (UID: \"0be6285e-fbac-4d35-8888-431e53289a2f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-64wml" Jul 06 16:39:58.000129 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:57.999941 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0be6285e-fbac-4d35-8888-431e53289a2f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-64wml\" (UID: \"0be6285e-fbac-4d35-8888-431e53289a2f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-64wml" Jul 06 16:39:58.000669 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:58.000645 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0be6285e-fbac-4d35-8888-431e53289a2f-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-64wml\" (UID: \"0be6285e-fbac-4d35-8888-431e53289a2f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-64wml" Jul 06 16:39:58.002538 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:58.002515 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0be6285e-fbac-4d35-8888-431e53289a2f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-64wml\" (UID: \"0be6285e-fbac-4d35-8888-431e53289a2f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-64wml" Jul 06 16:39:58.002626 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:58.002536 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/0be6285e-fbac-4d35-8888-431e53289a2f-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-64wml\" (UID: \"0be6285e-fbac-4d35-8888-431e53289a2f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-64wml" Jul 06 16:39:58.019856 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:58.019831 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jg8b\" (UniqueName: \"kubernetes.io/projected/0be6285e-fbac-4d35-8888-431e53289a2f-kube-api-access-4jg8b\") pod \"prometheus-operator-567f7b8b7c-64wml\" (UID: \"0be6285e-fbac-4d35-8888-431e53289a2f\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-64wml" Jul 06 16:39:58.083802 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:58.083777 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-64wml" Jul 06 16:39:58.221307 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:58.221275 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-64wml"] Jul 06 16:39:58.224089 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:39:58.224058 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0be6285e_fbac_4d35_8888_431e53289a2f.slice/crio-565d2a34f479379a04c504d4fc5ef09da23e23b92a4e02ec2ec82e1a3544fa0d WatchSource:0}: Error finding container 565d2a34f479379a04c504d4fc5ef09da23e23b92a4e02ec2ec82e1a3544fa0d: Status 404 returned error can't find the container with id 565d2a34f479379a04c504d4fc5ef09da23e23b92a4e02ec2ec82e1a3544fa0d Jul 06 16:39:58.691111 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:58.691075 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-87b54f4c7-4lpwt_f7221e3e-0a2d-4d91-b0f1-1b303255b302/router/0.log" Jul 06 16:39:58.742428 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:58.742395 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-64wml" event={"ID":"0be6285e-fbac-4d35-8888-431e53289a2f","Type":"ContainerStarted","Data":"565d2a34f479379a04c504d4fc5ef09da23e23b92a4e02ec2ec82e1a3544fa0d"} Jul 06 16:39:59.099954 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:59.099918 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4xll7_f13c431e-6bae-4714-a1ba-49fa1c767244/serve-healthcheck-canary/0.log" Jul 06 16:39:59.499006 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:59.498917 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-56fcf86bf-ht4xk_36d6010c-f9cd-4fea-9e77-3cf687d3616b/cluster-samples-operator/0.log" Jul 06 16:39:59.704723 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:39:59.704688 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-56fcf86bf-ht4xk_36d6010c-f9cd-4fea-9e77-3cf687d3616b/cluster-samples-operator-watch/0.log" Jul 06 16:40:00.751447 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:00.751411 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-64wml" event={"ID":"0be6285e-fbac-4d35-8888-431e53289a2f","Type":"ContainerStarted","Data":"7ebb9993aab8042ddbb39dbd56b5a52ff9c170ef934237b0b13d76fb71a47e99"} Jul 06 16:40:00.751862 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:00.751487 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-64wml" event={"ID":"0be6285e-fbac-4d35-8888-431e53289a2f","Type":"ContainerStarted","Data":"331bc15d0b48f197ad6fe6813972f389154db6e62eb9609e03b6e18f2edd5671"} Jul 06 16:40:00.797838 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:00.797789 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-64wml" podStartSLOduration=2.192127239 podStartE2EDuration="3.79777155s" totalImagesPullingTime="1.605644311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:40:00.796771028 +0000 UTC m=+114.974630152" watchObservedRunningTime="2026-07-06 16:40:00.79777155 +0000 UTC m=+114.975630661" Jul 06 16:40:03.756619 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.756586 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-v4bch"] Jul 06 16:40:03.767445 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.767416 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.775915 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.775877 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 16:40:03.776060 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.775879 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 16:40:03.776060 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.776055 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 16:40:03.776548 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.776525 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-64ncg\"" Jul 06 16:40:03.850909 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.850871 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/173ca302-b638-42e3-891e-467c6a727dd9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.850909 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.850915 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/173ca302-b638-42e3-891e-467c6a727dd9-node-exporter-tls\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.851154 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.850937 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/173ca302-b638-42e3-891e-467c6a727dd9-metrics-client-ca\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.851154 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.851018 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/173ca302-b638-42e3-891e-467c6a727dd9-node-exporter-accelerators-collector-config\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.851154 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.851066 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/173ca302-b638-42e3-891e-467c6a727dd9-sys\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.851154 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.851083 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/173ca302-b638-42e3-891e-467c6a727dd9-node-exporter-wtmp\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.851154 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.851143 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvl5f\" (UniqueName: \"kubernetes.io/projected/173ca302-b638-42e3-891e-467c6a727dd9-kube-api-access-rvl5f\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.851377 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.851195 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/173ca302-b638-42e3-891e-467c6a727dd9-node-exporter-textfile\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.851377 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.851266 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/173ca302-b638-42e3-891e-467c6a727dd9-root\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.951788 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.951753 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/173ca302-b638-42e3-891e-467c6a727dd9-metrics-client-ca\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.951788 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.951790 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/173ca302-b638-42e3-891e-467c6a727dd9-node-exporter-accelerators-collector-config\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.951993 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.951821 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/173ca302-b638-42e3-891e-467c6a727dd9-sys\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.951993 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.951838 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/173ca302-b638-42e3-891e-467c6a727dd9-node-exporter-wtmp\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.951993 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.951881 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvl5f\" (UniqueName: \"kubernetes.io/projected/173ca302-b638-42e3-891e-467c6a727dd9-kube-api-access-rvl5f\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.951993 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.951898 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/173ca302-b638-42e3-891e-467c6a727dd9-sys\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.951993 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.951915 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/173ca302-b638-42e3-891e-467c6a727dd9-node-exporter-textfile\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.952240 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.952030 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/173ca302-b638-42e3-891e-467c6a727dd9-root\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.952240 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.952087 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/173ca302-b638-42e3-891e-467c6a727dd9-root\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.952240 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.952088 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/173ca302-b638-42e3-891e-467c6a727dd9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.952240 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.952038 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/173ca302-b638-42e3-891e-467c6a727dd9-node-exporter-wtmp\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.952240 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.952157 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/173ca302-b638-42e3-891e-467c6a727dd9-node-exporter-tls\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.952497 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.952280 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/173ca302-b638-42e3-891e-467c6a727dd9-node-exporter-textfile\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.952497 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.952457 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/173ca302-b638-42e3-891e-467c6a727dd9-node-exporter-accelerators-collector-config\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.952594 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.952499 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/173ca302-b638-42e3-891e-467c6a727dd9-metrics-client-ca\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.954380 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.954353 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/173ca302-b638-42e3-891e-467c6a727dd9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.954499 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.954463 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/173ca302-b638-42e3-891e-467c6a727dd9-node-exporter-tls\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:03.986257 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:03.986226 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvl5f\" (UniqueName: \"kubernetes.io/projected/173ca302-b638-42e3-891e-467c6a727dd9-kube-api-access-rvl5f\") pod \"node-exporter-v4bch\" (UID: \"173ca302-b638-42e3-891e-467c6a727dd9\") " pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:04.080008 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:04.079924 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-v4bch" Jul 06 16:40:04.091581 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:40:04.091552 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod173ca302_b638_42e3_891e_467c6a727dd9.slice/crio-fc1b2ec7b9642250fcc042e03cd30e37f12d223cb72d47f7da1e77d34b417dae WatchSource:0}: Error finding container fc1b2ec7b9642250fcc042e03cd30e37f12d223cb72d47f7da1e77d34b417dae: Status 404 returned error can't find the container with id fc1b2ec7b9642250fcc042e03cd30e37f12d223cb72d47f7da1e77d34b417dae Jul 06 16:40:04.769756 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:04.769724 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v4bch" event={"ID":"173ca302-b638-42e3-891e-467c6a727dd9","Type":"ContainerStarted","Data":"fc1b2ec7b9642250fcc042e03cd30e37f12d223cb72d47f7da1e77d34b417dae"} Jul 06 16:40:05.775042 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:05.774960 2580 generic.go:356] "Generic (PLEG): container finished" podID="173ca302-b638-42e3-891e-467c6a727dd9" containerID="971e52061ef94f8b26a01fdbefde0096c4ee153cf47e5067778efa7c279194ba" exitCode=0 Jul 06 16:40:05.775042 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:05.775000 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v4bch" event={"ID":"173ca302-b638-42e3-891e-467c6a727dd9","Type":"ContainerDied","Data":"971e52061ef94f8b26a01fdbefde0096c4ee153cf47e5067778efa7c279194ba"} Jul 06 16:40:06.781186 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:06.781142 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v4bch" event={"ID":"173ca302-b638-42e3-891e-467c6a727dd9","Type":"ContainerStarted","Data":"8cedd98755d95abd30ffcc99db54e4498ef352d5e2fc9956b5f5ff1f6062dbb4"} Jul 06 16:40:06.781585 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:06.781254 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v4bch" event={"ID":"173ca302-b638-42e3-891e-467c6a727dd9","Type":"ContainerStarted","Data":"d42a9d9f82845ef52098a3d43f8925d84be28030d0ef81228b47196f750b2d8e"} Jul 06 16:40:06.812925 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:06.812870 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-v4bch" podStartSLOduration=2.476269488 podStartE2EDuration="3.812854186s" totalImagesPullingTime="1.273617568s" totalInitContainerRuntime="62.96713ms" isStatefulPod=true podCreationTimestamp="2026-07-06 16:40:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:40:06.811759952 +0000 UTC m=+120.989619075" watchObservedRunningTime="2026-07-06 16:40:06.812854186 +0000 UTC m=+120.990713297" Jul 06 16:40:08.066460 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.066425 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-747b755857-hhwcw"] Jul 06 16:40:08.072870 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.072847 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.107676 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.107177 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 06 16:40:08.107676 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.107335 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 06 16:40:08.107676 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.107177 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 06 16:40:08.107676 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.107627 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 06 16:40:08.109031 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.108408 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-8tpr2g60pmr1a\"" Jul 06 16:40:08.109031 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.108691 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 06 16:40:08.109031 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.108879 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-rmfr6\"" Jul 06 16:40:08.136872 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.136841 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-747b755857-hhwcw"] Jul 06 16:40:08.189361 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.189328 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e136234c-51b1-45cf-8127-f4585f548774-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.189539 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.189375 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e136234c-51b1-45cf-8127-f4585f548774-secret-grpc-tls\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.189539 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.189404 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e136234c-51b1-45cf-8127-f4585f548774-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.189539 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.189511 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e136234c-51b1-45cf-8127-f4585f548774-metrics-client-ca\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.189651 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.189551 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e136234c-51b1-45cf-8127-f4585f548774-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.189651 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.189579 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dhbp\" (UniqueName: \"kubernetes.io/projected/e136234c-51b1-45cf-8127-f4585f548774-kube-api-access-9dhbp\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.189651 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.189647 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e136234c-51b1-45cf-8127-f4585f548774-secret-thanos-querier-tls\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.189735 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.189666 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e136234c-51b1-45cf-8127-f4585f548774-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.274884 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.274854 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-cnv9g"] Jul 06 16:40:08.277984 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.277964 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cnv9g" Jul 06 16:40:08.284104 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.284079 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 06 16:40:08.284986 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.284965 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-5jnl2\"" Jul 06 16:40:08.290259 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.290233 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e136234c-51b1-45cf-8127-f4585f548774-secret-grpc-tls\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.290378 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.290272 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e136234c-51b1-45cf-8127-f4585f548774-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.290378 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.290302 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e136234c-51b1-45cf-8127-f4585f548774-metrics-client-ca\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.290378 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.290328 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e136234c-51b1-45cf-8127-f4585f548774-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.290378 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.290348 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dhbp\" (UniqueName: \"kubernetes.io/projected/e136234c-51b1-45cf-8127-f4585f548774-kube-api-access-9dhbp\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.290618 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.290426 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e136234c-51b1-45cf-8127-f4585f548774-secret-thanos-querier-tls\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.290618 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.290452 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e136234c-51b1-45cf-8127-f4585f548774-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.290618 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.290503 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e136234c-51b1-45cf-8127-f4585f548774-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.291080 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.291059 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e136234c-51b1-45cf-8127-f4585f548774-metrics-client-ca\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.293317 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.293289 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e136234c-51b1-45cf-8127-f4585f548774-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.293498 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.293439 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e136234c-51b1-45cf-8127-f4585f548774-secret-grpc-tls\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.293586 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.293493 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e136234c-51b1-45cf-8127-f4585f548774-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.293586 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.293509 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/e136234c-51b1-45cf-8127-f4585f548774-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.293882 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.293854 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/e136234c-51b1-45cf-8127-f4585f548774-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.294048 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.294027 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/e136234c-51b1-45cf-8127-f4585f548774-secret-thanos-querier-tls\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.338220 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.338185 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dhbp\" (UniqueName: \"kubernetes.io/projected/e136234c-51b1-45cf-8127-f4585f548774-kube-api-access-9dhbp\") pod \"thanos-querier-747b755857-hhwcw\" (UID: \"e136234c-51b1-45cf-8127-f4585f548774\") " pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.383038 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.383006 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:08.391426 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.391400 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6tmq\" (UniqueName: \"kubernetes.io/projected/2fb77d8e-423d-41f9-90ec-a9f18b514ea5-kube-api-access-d6tmq\") pod \"monitoring-plugin-d5dcff7cf-cnv9g\" (UID: \"2fb77d8e-423d-41f9-90ec-a9f18b514ea5\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cnv9g" Jul 06 16:40:08.391581 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.391442 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2fb77d8e-423d-41f9-90ec-a9f18b514ea5-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-cnv9g\" (UID: \"2fb77d8e-423d-41f9-90ec-a9f18b514ea5\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cnv9g" Jul 06 16:40:08.408597 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.408071 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-cnv9g"] Jul 06 16:40:08.492674 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.492529 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6tmq\" (UniqueName: \"kubernetes.io/projected/2fb77d8e-423d-41f9-90ec-a9f18b514ea5-kube-api-access-d6tmq\") pod \"monitoring-plugin-d5dcff7cf-cnv9g\" (UID: \"2fb77d8e-423d-41f9-90ec-a9f18b514ea5\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cnv9g" Jul 06 16:40:08.492674 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.492593 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2fb77d8e-423d-41f9-90ec-a9f18b514ea5-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-cnv9g\" (UID: \"2fb77d8e-423d-41f9-90ec-a9f18b514ea5\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cnv9g" Jul 06 16:40:08.492809 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:40:08.492744 2580 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 06 16:40:08.492809 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:40:08.492807 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fb77d8e-423d-41f9-90ec-a9f18b514ea5-monitoring-plugin-cert podName:2fb77d8e-423d-41f9-90ec-a9f18b514ea5 nodeName:}" failed. No retries permitted until 2026-07-06 16:40:08.992788217 +0000 UTC m=+123.170647311 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/2fb77d8e-423d-41f9-90ec-a9f18b514ea5-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-cnv9g" (UID: "2fb77d8e-423d-41f9-90ec-a9f18b514ea5") : secret "monitoring-plugin-cert" not found Jul 06 16:40:08.551177 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.551145 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6tmq\" (UniqueName: \"kubernetes.io/projected/2fb77d8e-423d-41f9-90ec-a9f18b514ea5-kube-api-access-d6tmq\") pod \"monitoring-plugin-d5dcff7cf-cnv9g\" (UID: \"2fb77d8e-423d-41f9-90ec-a9f18b514ea5\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cnv9g" Jul 06 16:40:08.631419 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.631389 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-747b755857-hhwcw"] Jul 06 16:40:08.633349 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:40:08.633314 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode136234c_51b1_45cf_8127_f4585f548774.slice/crio-756e4f709b187f6667e056c0c4df051e615477984465bfbb946f5d0ff71f7f3f WatchSource:0}: Error finding container 756e4f709b187f6667e056c0c4df051e615477984465bfbb946f5d0ff71f7f3f: Status 404 returned error can't find the container with id 756e4f709b187f6667e056c0c4df051e615477984465bfbb946f5d0ff71f7f3f Jul 06 16:40:08.793021 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.792990 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" event={"ID":"e136234c-51b1-45cf-8127-f4585f548774","Type":"ContainerStarted","Data":"756e4f709b187f6667e056c0c4df051e615477984465bfbb946f5d0ff71f7f3f"} Jul 06 16:40:08.997040 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.996941 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2fb77d8e-423d-41f9-90ec-a9f18b514ea5-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-cnv9g\" (UID: \"2fb77d8e-423d-41f9-90ec-a9f18b514ea5\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cnv9g" Jul 06 16:40:08.999527 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:08.999495 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2fb77d8e-423d-41f9-90ec-a9f18b514ea5-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-cnv9g\" (UID: \"2fb77d8e-423d-41f9-90ec-a9f18b514ea5\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cnv9g" Jul 06 16:40:09.187615 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:09.187572 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cnv9g" Jul 06 16:40:09.503617 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:09.503583 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-cnv9g"] Jul 06 16:40:09.521460 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:40:09.521434 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fb77d8e_423d_41f9_90ec_a9f18b514ea5.slice/crio-a38d75607ad535096c1b1c07cfdd61a271c9c2991aeac253116df223830b2707 WatchSource:0}: Error finding container a38d75607ad535096c1b1c07cfdd61a271c9c2991aeac253116df223830b2707: Status 404 returned error can't find the container with id a38d75607ad535096c1b1c07cfdd61a271c9c2991aeac253116df223830b2707 Jul 06 16:40:09.798181 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:09.798143 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cnv9g" event={"ID":"2fb77d8e-423d-41f9-90ec-a9f18b514ea5","Type":"ContainerStarted","Data":"a38d75607ad535096c1b1c07cfdd61a271c9c2991aeac253116df223830b2707"} Jul 06 16:40:10.803566 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:10.803534 2580 generic.go:356] "Generic (PLEG): container finished" podID="861b3db1-e952-457e-abdd-d36829821c19" containerID="078d30bdc84414bfc3ef69cfb944003b27d4c29cdc5bbde548ddcd5b27c6dc19" exitCode=0 Jul 06 16:40:10.803945 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:10.803576 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-7dtv6" event={"ID":"861b3db1-e952-457e-abdd-d36829821c19","Type":"ContainerDied","Data":"078d30bdc84414bfc3ef69cfb944003b27d4c29cdc5bbde548ddcd5b27c6dc19"} Jul 06 16:40:10.803945 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:10.803849 2580 scope.go:117] "RemoveContainer" containerID="078d30bdc84414bfc3ef69cfb944003b27d4c29cdc5bbde548ddcd5b27c6dc19" Jul 06 16:40:11.810830 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:11.810798 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-7dtv6" event={"ID":"861b3db1-e952-457e-abdd-d36829821c19","Type":"ContainerStarted","Data":"c91f94e94a29862a79f7eb19f172338a0c7291516f92856dd381051ef618cd9c"} Jul 06 16:40:12.144636 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.144597 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" podUID="18464d2a-4b04-486f-bfc2-ac4d21f31578" containerName="registry" containerID="cri-o://af1502ac2982ba50cf5661e4bf082c68aabafee86038d66c6b7302e3b46b943e" gracePeriod=30 Jul 06 16:40:12.423556 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.423535 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:40:12.535064 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.535037 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/18464d2a-4b04-486f-bfc2-ac4d21f31578-ca-trust-extracted\") pod \"18464d2a-4b04-486f-bfc2-ac4d21f31578\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " Jul 06 16:40:12.535185 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.535079 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbh85\" (UniqueName: \"kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-kube-api-access-sbh85\") pod \"18464d2a-4b04-486f-bfc2-ac4d21f31578\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " Jul 06 16:40:12.535185 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.535109 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/18464d2a-4b04-486f-bfc2-ac4d21f31578-installation-pull-secrets\") pod \"18464d2a-4b04-486f-bfc2-ac4d21f31578\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " Jul 06 16:40:12.535185 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.535165 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/18464d2a-4b04-486f-bfc2-ac4d21f31578-trusted-ca\") pod \"18464d2a-4b04-486f-bfc2-ac4d21f31578\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " Jul 06 16:40:12.535346 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.535204 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-bound-sa-token\") pod \"18464d2a-4b04-486f-bfc2-ac4d21f31578\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " Jul 06 16:40:12.535346 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.535245 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/18464d2a-4b04-486f-bfc2-ac4d21f31578-image-registry-private-configuration\") pod \"18464d2a-4b04-486f-bfc2-ac4d21f31578\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " Jul 06 16:40:12.535346 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.535274 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-certificates\") pod \"18464d2a-4b04-486f-bfc2-ac4d21f31578\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " Jul 06 16:40:12.535346 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.535332 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls\") pod \"18464d2a-4b04-486f-bfc2-ac4d21f31578\" (UID: \"18464d2a-4b04-486f-bfc2-ac4d21f31578\") " Jul 06 16:40:12.535649 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.535614 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18464d2a-4b04-486f-bfc2-ac4d21f31578-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "18464d2a-4b04-486f-bfc2-ac4d21f31578" (UID: "18464d2a-4b04-486f-bfc2-ac4d21f31578"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:40:12.535873 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.535840 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "18464d2a-4b04-486f-bfc2-ac4d21f31578" (UID: "18464d2a-4b04-486f-bfc2-ac4d21f31578"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 16:40:12.537545 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.537505 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-kube-api-access-sbh85" (OuterVolumeSpecName: "kube-api-access-sbh85") pod "18464d2a-4b04-486f-bfc2-ac4d21f31578" (UID: "18464d2a-4b04-486f-bfc2-ac4d21f31578"). InnerVolumeSpecName "kube-api-access-sbh85". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:40:12.537666 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.537643 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18464d2a-4b04-486f-bfc2-ac4d21f31578-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "18464d2a-4b04-486f-bfc2-ac4d21f31578" (UID: "18464d2a-4b04-486f-bfc2-ac4d21f31578"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:40:12.537849 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.537825 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "18464d2a-4b04-486f-bfc2-ac4d21f31578" (UID: "18464d2a-4b04-486f-bfc2-ac4d21f31578"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:40:12.537937 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.537858 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18464d2a-4b04-486f-bfc2-ac4d21f31578-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "18464d2a-4b04-486f-bfc2-ac4d21f31578" (UID: "18464d2a-4b04-486f-bfc2-ac4d21f31578"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:40:12.538329 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.538303 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "18464d2a-4b04-486f-bfc2-ac4d21f31578" (UID: "18464d2a-4b04-486f-bfc2-ac4d21f31578"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:40:12.543628 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.543607 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18464d2a-4b04-486f-bfc2-ac4d21f31578-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "18464d2a-4b04-486f-bfc2-ac4d21f31578" (UID: "18464d2a-4b04-486f-bfc2-ac4d21f31578"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:40:12.636775 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.636751 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/18464d2a-4b04-486f-bfc2-ac4d21f31578-trusted-ca\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:40:12.636775 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.636772 2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-bound-sa-token\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:40:12.636908 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.636782 2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/18464d2a-4b04-486f-bfc2-ac4d21f31578-image-registry-private-configuration\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:40:12.636908 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.636791 2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-certificates\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:40:12.636908 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.636801 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-registry-tls\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:40:12.636908 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.636809 2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/18464d2a-4b04-486f-bfc2-ac4d21f31578-ca-trust-extracted\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:40:12.636908 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.636817 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbh85\" (UniqueName: \"kubernetes.io/projected/18464d2a-4b04-486f-bfc2-ac4d21f31578-kube-api-access-sbh85\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:40:12.636908 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.636825 2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/18464d2a-4b04-486f-bfc2-ac4d21f31578-installation-pull-secrets\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:40:12.815987 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.815890 2580 generic.go:356] "Generic (PLEG): container finished" podID="18464d2a-4b04-486f-bfc2-ac4d21f31578" containerID="af1502ac2982ba50cf5661e4bf082c68aabafee86038d66c6b7302e3b46b943e" exitCode=0 Jul 06 16:40:12.815987 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.815937 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" Jul 06 16:40:12.815987 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.815940 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" event={"ID":"18464d2a-4b04-486f-bfc2-ac4d21f31578","Type":"ContainerDied","Data":"af1502ac2982ba50cf5661e4bf082c68aabafee86038d66c6b7302e3b46b943e"} Jul 06 16:40:12.816461 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.815989 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c58c78ccb-5sns6" event={"ID":"18464d2a-4b04-486f-bfc2-ac4d21f31578","Type":"ContainerDied","Data":"dae2b0d17603bee1437f3ecd022f77c4b444f7d0a9e01c3a2a3628f537f38290"} Jul 06 16:40:12.816461 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.816014 2580 scope.go:117] "RemoveContainer" containerID="af1502ac2982ba50cf5661e4bf082c68aabafee86038d66c6b7302e3b46b943e" Jul 06 16:40:12.817676 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.817649 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cnv9g" event={"ID":"2fb77d8e-423d-41f9-90ec-a9f18b514ea5","Type":"ContainerStarted","Data":"445113d8e9a92d7a6741d12363a5cf243473cacd93d7edc37438c08ebf5f35c6"} Jul 06 16:40:12.817824 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.817802 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cnv9g" Jul 06 16:40:12.820161 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.820139 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" event={"ID":"e136234c-51b1-45cf-8127-f4585f548774","Type":"ContainerStarted","Data":"38a18c18dadd3d53626172707fbfec01500e2ec8a5a72dab7ed3ef7e7d2bd53c"} Jul 06 16:40:12.820243 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.820193 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" event={"ID":"e136234c-51b1-45cf-8127-f4585f548774","Type":"ContainerStarted","Data":"f5282429403940690e794ac4529721e00abe9ab7c29b8fa80722940f746ce425"} Jul 06 16:40:12.820243 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.820227 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" event={"ID":"e136234c-51b1-45cf-8127-f4585f548774","Type":"ContainerStarted","Data":"2111b155302efa1507652363a4aec0fd2e587ace8dd0e56637403872e0cfa88c"} Jul 06 16:40:12.824309 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.824286 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cnv9g" Jul 06 16:40:12.825086 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.825056 2580 scope.go:117] "RemoveContainer" containerID="af1502ac2982ba50cf5661e4bf082c68aabafee86038d66c6b7302e3b46b943e" Jul 06 16:40:12.825338 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:40:12.825317 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af1502ac2982ba50cf5661e4bf082c68aabafee86038d66c6b7302e3b46b943e\": container with ID starting with af1502ac2982ba50cf5661e4bf082c68aabafee86038d66c6b7302e3b46b943e not found: ID does not exist" containerID="af1502ac2982ba50cf5661e4bf082c68aabafee86038d66c6b7302e3b46b943e" Jul 06 16:40:12.825400 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.825346 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af1502ac2982ba50cf5661e4bf082c68aabafee86038d66c6b7302e3b46b943e"} err="failed to get container status \"af1502ac2982ba50cf5661e4bf082c68aabafee86038d66c6b7302e3b46b943e\": rpc error: code = NotFound desc = could not find container \"af1502ac2982ba50cf5661e4bf082c68aabafee86038d66c6b7302e3b46b943e\": container with ID starting with af1502ac2982ba50cf5661e4bf082c68aabafee86038d66c6b7302e3b46b943e not found: ID does not exist" Jul 06 16:40:12.861245 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.861201 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cnv9g" podStartSLOduration=2.107958398 podStartE2EDuration="4.861187625s" totalImagesPullingTime="2.753229227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:40:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:40:12.859068872 +0000 UTC m=+127.036927985" watchObservedRunningTime="2026-07-06 16:40:12.861187625 +0000 UTC m=+127.039046737" Jul 06 16:40:12.930362 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.930333 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7c58c78ccb-5sns6"] Jul 06 16:40:12.944533 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:12.944513 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7c58c78ccb-5sns6"] Jul 06 16:40:13.827314 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:13.827242 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" event={"ID":"e136234c-51b1-45cf-8127-f4585f548774","Type":"ContainerStarted","Data":"bbd57b398dd87b7a5f7328878833fb72df10f35b2026af8848eb2949096b3172"} Jul 06 16:40:13.827732 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:13.827338 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" event={"ID":"e136234c-51b1-45cf-8127-f4585f548774","Type":"ContainerStarted","Data":"d481bf335437d3c304258c8b377e7a8616074b6640e1e98083a9684b209513f0"} Jul 06 16:40:13.827732 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:13.827371 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" event={"ID":"e136234c-51b1-45cf-8127-f4585f548774","Type":"ContainerStarted","Data":"67132f799d827ed3aa1b7f999b7219c278680d51f479e1542ec212dfbb9b766e"} Jul 06 16:40:13.877954 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:13.877907 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" podStartSLOduration=1.2105915999999999 podStartE2EDuration="5.877891663s" totalImagesPullingTime="4.667300063s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:40:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:40:13.877112813 +0000 UTC m=+128.054971924" watchObservedRunningTime="2026-07-06 16:40:13.877891663 +0000 UTC m=+128.055750789" Jul 06 16:40:14.263944 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:14.263910 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18464d2a-4b04-486f-bfc2-ac4d21f31578" path="/var/lib/kubelet/pods/18464d2a-4b04-486f-bfc2-ac4d21f31578/volumes" Jul 06 16:40:14.831445 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:14.831415 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:16.060606 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:16.060572 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs\") pod \"network-metrics-daemon-5m692\" (UID: \"3f867a4a-dbf4-46a7-a79b-c30d916fa0f8\") " pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:40:16.062845 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:16.062828 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f867a4a-dbf4-46a7-a79b-c30d916fa0f8-metrics-certs\") pod \"network-metrics-daemon-5m692\" (UID: \"3f867a4a-dbf4-46a7-a79b-c30d916fa0f8\") " pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:40:16.296460 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:16.296433 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ntdtl\"" Jul 06 16:40:16.300509 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:16.300495 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5m692" Jul 06 16:40:16.653014 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:16.652977 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5m692"] Jul 06 16:40:16.669886 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:40:16.669845 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f867a4a_dbf4_46a7_a79b_c30d916fa0f8.slice/crio-9d9beb63fc2a163751e175060dc30a1b1416bd124f0566dc716207f0ff85867b WatchSource:0}: Error finding container 9d9beb63fc2a163751e175060dc30a1b1416bd124f0566dc716207f0ff85867b: Status 404 returned error can't find the container with id 9d9beb63fc2a163751e175060dc30a1b1416bd124f0566dc716207f0ff85867b Jul 06 16:40:16.839317 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:16.839264 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5m692" event={"ID":"3f867a4a-dbf4-46a7-a79b-c30d916fa0f8","Type":"ContainerStarted","Data":"9d9beb63fc2a163751e175060dc30a1b1416bd124f0566dc716207f0ff85867b"} Jul 06 16:40:17.845077 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:17.845044 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5m692" event={"ID":"3f867a4a-dbf4-46a7-a79b-c30d916fa0f8","Type":"ContainerStarted","Data":"fbeb27e65232996c6f55b0d3d451c5db62613a7dde6e0b520606339195e66f33"} Jul 06 16:40:17.845428 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:17.845107 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5m692" event={"ID":"3f867a4a-dbf4-46a7-a79b-c30d916fa0f8","Type":"ContainerStarted","Data":"adf09f1758002120b5e70e9c7657bb62ca8059ede7c517c127b941908b5362c5"} Jul 06 16:40:20.839731 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:20.839701 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-747b755857-hhwcw" Jul 06 16:40:20.914533 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:20.914482 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5m692" podStartSLOduration=133.922450839 podStartE2EDuration="2m14.914433164s" totalImagesPullingTime="991.982325ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:40:17.885361843 +0000 UTC m=+132.063220954" watchObservedRunningTime="2026-07-06 16:40:20.914433164 +0000 UTC m=+135.092292273" Jul 06 16:40:30.896528 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:30.896495 2580 generic.go:356] "Generic (PLEG): container finished" podID="2372b154-c850-4003-ba8a-3534dd9c8cde" containerID="6d906295715f1cd2c8bcc81a192acf45829bf8f2e834cf4ecf64558704a55a1c" exitCode=0 Jul 06 16:40:30.896902 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:30.896546 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4ggmp" event={"ID":"2372b154-c850-4003-ba8a-3534dd9c8cde","Type":"ContainerDied","Data":"6d906295715f1cd2c8bcc81a192acf45829bf8f2e834cf4ecf64558704a55a1c"} Jul 06 16:40:30.896902 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:30.896810 2580 scope.go:117] "RemoveContainer" containerID="6d906295715f1cd2c8bcc81a192acf45829bf8f2e834cf4ecf64558704a55a1c" Jul 06 16:40:31.902020 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:40:31.901991 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-4ggmp" event={"ID":"2372b154-c850-4003-ba8a-3534dd9c8cde","Type":"ContainerStarted","Data":"05ec6d5dce018d16abf5a1a24cc3286951f3fad5b75c006311342cbc6d56d879"} Jul 06 16:43:06.166455 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:43:06.166427 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 16:43:06.171430 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:43:06.171405 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 16:43:06.174149 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:43:06.174077 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 16:43:06.180056 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:43:06.180034 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 16:43:06.180269 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:43:06.180250 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 16:47:43.863491 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:43.863438 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd"] Jul 06 16:47:43.864117 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:43.863882 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18464d2a-4b04-486f-bfc2-ac4d21f31578" containerName="registry" Jul 06 16:47:43.864117 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:43.863901 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="18464d2a-4b04-486f-bfc2-ac4d21f31578" containerName="registry" Jul 06 16:47:43.864117 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:43.863993 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="18464d2a-4b04-486f-bfc2-ac4d21f31578" containerName="registry" Jul 06 16:47:43.867081 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:43.867061 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd" Jul 06 16:47:43.869703 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:43.869684 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 16:47:43.870089 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:43.870070 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mxvhm\"" Jul 06 16:47:43.870182 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:43.870149 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 16:47:43.888898 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:43.888870 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd"] Jul 06 16:47:43.955752 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:43.955717 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2945bd3-9da4-4454-b637-13a008f51354-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd\" (UID: \"f2945bd3-9da4-4454-b637-13a008f51354\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd" Jul 06 16:47:43.955929 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:43.955771 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2945bd3-9da4-4454-b637-13a008f51354-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd\" (UID: \"f2945bd3-9da4-4454-b637-13a008f51354\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd" Jul 06 16:47:43.955929 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:43.955798 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf58d\" (UniqueName: \"kubernetes.io/projected/f2945bd3-9da4-4454-b637-13a008f51354-kube-api-access-hf58d\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd\" (UID: \"f2945bd3-9da4-4454-b637-13a008f51354\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd" Jul 06 16:47:44.057019 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:44.056985 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2945bd3-9da4-4454-b637-13a008f51354-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd\" (UID: \"f2945bd3-9da4-4454-b637-13a008f51354\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd" Jul 06 16:47:44.057132 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:44.057030 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf58d\" (UniqueName: \"kubernetes.io/projected/f2945bd3-9da4-4454-b637-13a008f51354-kube-api-access-hf58d\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd\" (UID: \"f2945bd3-9da4-4454-b637-13a008f51354\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd" Jul 06 16:47:44.057132 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:44.057076 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2945bd3-9da4-4454-b637-13a008f51354-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd\" (UID: \"f2945bd3-9da4-4454-b637-13a008f51354\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd" Jul 06 16:47:44.057379 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:44.057360 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2945bd3-9da4-4454-b637-13a008f51354-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd\" (UID: \"f2945bd3-9da4-4454-b637-13a008f51354\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd" Jul 06 16:47:44.057414 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:44.057386 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2945bd3-9da4-4454-b637-13a008f51354-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd\" (UID: \"f2945bd3-9da4-4454-b637-13a008f51354\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd" Jul 06 16:47:44.069937 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:44.069911 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf58d\" (UniqueName: \"kubernetes.io/projected/f2945bd3-9da4-4454-b637-13a008f51354-kube-api-access-hf58d\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd\" (UID: \"f2945bd3-9da4-4454-b637-13a008f51354\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd" Jul 06 16:47:44.176036 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:44.175948 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd" Jul 06 16:47:44.309708 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:44.309685 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd"] Jul 06 16:47:44.312121 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:47:44.312093 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2945bd3_9da4_4454_b637_13a008f51354.slice/crio-03aecc2da93197054965cdb4022a498cbc0f3b6525e8a76db4d5bca6d4dab19f WatchSource:0}: Error finding container 03aecc2da93197054965cdb4022a498cbc0f3b6525e8a76db4d5bca6d4dab19f: Status 404 returned error can't find the container with id 03aecc2da93197054965cdb4022a498cbc0f3b6525e8a76db4d5bca6d4dab19f Jul 06 16:47:44.314000 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:44.313980 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 16:47:44.505062 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:44.504973 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd" event={"ID":"f2945bd3-9da4-4454-b637-13a008f51354","Type":"ContainerStarted","Data":"03aecc2da93197054965cdb4022a498cbc0f3b6525e8a76db4d5bca6d4dab19f"} Jul 06 16:47:52.539711 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:52.539680 2580 generic.go:356] "Generic (PLEG): container finished" podID="f2945bd3-9da4-4454-b637-13a008f51354" containerID="8e32069e3a7c35f3d85d6306a8b94314655a41933270a84e1ac15a1d02bc6b53" exitCode=0 Jul 06 16:47:52.540120 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:52.539775 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd" event={"ID":"f2945bd3-9da4-4454-b637-13a008f51354","Type":"ContainerDied","Data":"8e32069e3a7c35f3d85d6306a8b94314655a41933270a84e1ac15a1d02bc6b53"} Jul 06 16:47:55.552602 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:55.552567 2580 generic.go:356] "Generic (PLEG): container finished" podID="f2945bd3-9da4-4454-b637-13a008f51354" containerID="a41551ce043f66e64664d410f4e91639ddb9f0209d0a33e7260d40f346a5b32b" exitCode=0 Jul 06 16:47:55.553008 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:47:55.552655 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd" event={"ID":"f2945bd3-9da4-4454-b637-13a008f51354","Type":"ContainerDied","Data":"a41551ce043f66e64664d410f4e91639ddb9f0209d0a33e7260d40f346a5b32b"} Jul 06 16:48:03.585869 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:03.585830 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd" event={"ID":"f2945bd3-9da4-4454-b637-13a008f51354","Type":"ContainerStarted","Data":"7f38361d750bf915a4d721c6a3f23e8e5829c76524460484bfc11368667533db"} Jul 06 16:48:03.623370 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:03.623323 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd" podStartSLOduration=2.605704427 podStartE2EDuration="20.623307775s" totalImagesPullingTime="18.017603348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:47:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:48:03.62300427 +0000 UTC m=+597.800863383" watchObservedRunningTime="2026-07-06 16:48:03.623307775 +0000 UTC m=+597.801166887" Jul 06 16:48:04.591421 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:04.591389 2580 generic.go:356] "Generic (PLEG): container finished" podID="f2945bd3-9da4-4454-b637-13a008f51354" containerID="7f38361d750bf915a4d721c6a3f23e8e5829c76524460484bfc11368667533db" exitCode=0 Jul 06 16:48:04.591884 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:04.591438 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd" event={"ID":"f2945bd3-9da4-4454-b637-13a008f51354","Type":"ContainerDied","Data":"7f38361d750bf915a4d721c6a3f23e8e5829c76524460484bfc11368667533db"} Jul 06 16:48:05.723013 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:05.722985 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd" Jul 06 16:48:05.838169 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:05.838111 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hf58d\" (UniqueName: \"kubernetes.io/projected/f2945bd3-9da4-4454-b637-13a008f51354-kube-api-access-hf58d\") pod \"f2945bd3-9da4-4454-b637-13a008f51354\" (UID: \"f2945bd3-9da4-4454-b637-13a008f51354\") " Jul 06 16:48:05.838373 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:05.838190 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2945bd3-9da4-4454-b637-13a008f51354-util\") pod \"f2945bd3-9da4-4454-b637-13a008f51354\" (UID: \"f2945bd3-9da4-4454-b637-13a008f51354\") " Jul 06 16:48:05.838373 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:05.838238 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2945bd3-9da4-4454-b637-13a008f51354-bundle\") pod \"f2945bd3-9da4-4454-b637-13a008f51354\" (UID: \"f2945bd3-9da4-4454-b637-13a008f51354\") " Jul 06 16:48:05.839201 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:05.839156 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2945bd3-9da4-4454-b637-13a008f51354-bundle" (OuterVolumeSpecName: "bundle") pod "f2945bd3-9da4-4454-b637-13a008f51354" (UID: "f2945bd3-9da4-4454-b637-13a008f51354"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:48:05.840662 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:05.840628 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2945bd3-9da4-4454-b637-13a008f51354-kube-api-access-hf58d" (OuterVolumeSpecName: "kube-api-access-hf58d") pod "f2945bd3-9da4-4454-b637-13a008f51354" (UID: "f2945bd3-9da4-4454-b637-13a008f51354"). InnerVolumeSpecName "kube-api-access-hf58d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:48:05.842816 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:05.842790 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2945bd3-9da4-4454-b637-13a008f51354-util" (OuterVolumeSpecName: "util") pod "f2945bd3-9da4-4454-b637-13a008f51354" (UID: "f2945bd3-9da4-4454-b637-13a008f51354"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:48:05.939831 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:05.939741 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2945bd3-9da4-4454-b637-13a008f51354-util\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:48:05.939831 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:05.939779 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2945bd3-9da4-4454-b637-13a008f51354-bundle\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:48:05.939831 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:05.939790 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hf58d\" (UniqueName: \"kubernetes.io/projected/f2945bd3-9da4-4454-b637-13a008f51354-kube-api-access-hf58d\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:48:06.198210 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:06.198128 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 16:48:06.199346 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:06.199328 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 16:48:06.202981 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:06.202961 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 16:48:06.204107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:06.204090 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 16:48:06.601203 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:06.601175 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd" Jul 06 16:48:06.601361 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:06.601182 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540qxhsd" event={"ID":"f2945bd3-9da4-4454-b637-13a008f51354","Type":"ContainerDied","Data":"03aecc2da93197054965cdb4022a498cbc0f3b6525e8a76db4d5bca6d4dab19f"} Jul 06 16:48:06.601361 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:06.601295 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="03aecc2da93197054965cdb4022a498cbc0f3b6525e8a76db4d5bca6d4dab19f" Jul 06 16:48:12.324165 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:12.324129 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxdz9"] Jul 06 16:48:12.324683 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:12.324583 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2945bd3-9da4-4454-b637-13a008f51354" containerName="util" Jul 06 16:48:12.324683 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:12.324603 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2945bd3-9da4-4454-b637-13a008f51354" containerName="util" Jul 06 16:48:12.324683 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:12.324614 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2945bd3-9da4-4454-b637-13a008f51354" containerName="pull" Jul 06 16:48:12.324683 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:12.324623 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2945bd3-9da4-4454-b637-13a008f51354" containerName="pull" Jul 06 16:48:12.324683 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:12.324635 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2945bd3-9da4-4454-b637-13a008f51354" containerName="extract" Jul 06 16:48:12.324683 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:12.324643 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2945bd3-9da4-4454-b637-13a008f51354" containerName="extract" Jul 06 16:48:12.324965 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:12.324724 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2945bd3-9da4-4454-b637-13a008f51354" containerName="extract" Jul 06 16:48:12.328093 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:12.328069 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxdz9" Jul 06 16:48:12.330851 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:12.330835 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-bpzm4\"" Jul 06 16:48:12.331270 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:12.331249 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 06 16:48:12.335385 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:12.335368 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 06 16:48:12.346047 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:12.346026 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxdz9"] Jul 06 16:48:12.392419 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:12.392387 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d72r\" (UniqueName: \"kubernetes.io/projected/23dc97be-3f63-45ba-93b7-40ca3bc01cec-kube-api-access-8d72r\") pod \"cert-manager-operator-controller-manager-55c444bf86-jxdz9\" (UID: \"23dc97be-3f63-45ba-93b7-40ca3bc01cec\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxdz9" Jul 06 16:48:12.392575 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:12.392424 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23dc97be-3f63-45ba-93b7-40ca3bc01cec-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jxdz9\" (UID: \"23dc97be-3f63-45ba-93b7-40ca3bc01cec\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxdz9" Jul 06 16:48:12.493021 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:12.492974 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d72r\" (UniqueName: \"kubernetes.io/projected/23dc97be-3f63-45ba-93b7-40ca3bc01cec-kube-api-access-8d72r\") pod \"cert-manager-operator-controller-manager-55c444bf86-jxdz9\" (UID: \"23dc97be-3f63-45ba-93b7-40ca3bc01cec\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxdz9" Jul 06 16:48:12.493021 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:12.493023 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23dc97be-3f63-45ba-93b7-40ca3bc01cec-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jxdz9\" (UID: \"23dc97be-3f63-45ba-93b7-40ca3bc01cec\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxdz9" Jul 06 16:48:12.493360 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:12.493344 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23dc97be-3f63-45ba-93b7-40ca3bc01cec-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jxdz9\" (UID: \"23dc97be-3f63-45ba-93b7-40ca3bc01cec\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxdz9" Jul 06 16:48:12.515867 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:12.515832 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d72r\" (UniqueName: \"kubernetes.io/projected/23dc97be-3f63-45ba-93b7-40ca3bc01cec-kube-api-access-8d72r\") pod \"cert-manager-operator-controller-manager-55c444bf86-jxdz9\" (UID: \"23dc97be-3f63-45ba-93b7-40ca3bc01cec\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxdz9" Jul 06 16:48:12.637044 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:12.637018 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxdz9" Jul 06 16:48:12.783309 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:12.783277 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxdz9"] Jul 06 16:48:12.786902 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:48:12.786875 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23dc97be_3f63_45ba_93b7_40ca3bc01cec.slice/crio-4151ce6da6b8ac0ef5e6ee3b3238644b5f66ecab18e6739c9d172f90368cf0b4 WatchSource:0}: Error finding container 4151ce6da6b8ac0ef5e6ee3b3238644b5f66ecab18e6739c9d172f90368cf0b4: Status 404 returned error can't find the container with id 4151ce6da6b8ac0ef5e6ee3b3238644b5f66ecab18e6739c9d172f90368cf0b4 Jul 06 16:48:13.630235 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:13.630199 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxdz9" event={"ID":"23dc97be-3f63-45ba-93b7-40ca3bc01cec","Type":"ContainerStarted","Data":"4151ce6da6b8ac0ef5e6ee3b3238644b5f66ecab18e6739c9d172f90368cf0b4"} Jul 06 16:48:15.643117 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:15.643082 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxdz9" event={"ID":"23dc97be-3f63-45ba-93b7-40ca3bc01cec","Type":"ContainerStarted","Data":"77f05a9a0ce706cecaa3f9d6a8fa6b9cdc64e990c3a48ef4ef2c53494ea70331"} Jul 06 16:48:15.705752 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:15.705688 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jxdz9" podStartSLOduration=1.5229376270000001 podStartE2EDuration="3.705666205s" totalImagesPullingTime="2.182728578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:48:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:48:15.702239505 +0000 UTC m=+609.880098616" watchObservedRunningTime="2026-07-06 16:48:15.705666205 +0000 UTC m=+609.883525319" Jul 06 16:48:17.032688 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:17.032653 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g"] Jul 06 16:48:17.036418 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:17.036396 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g" Jul 06 16:48:17.044045 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:17.044024 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 16:48:17.044158 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:17.044049 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 16:48:17.044221 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:17.044166 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mxvhm\"" Jul 06 16:48:17.129151 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:17.129119 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cqfk\" (UniqueName: \"kubernetes.io/projected/fe227383-0435-42f2-98a8-bd027d68d359-kube-api-access-6cqfk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g\" (UID: \"fe227383-0435-42f2-98a8-bd027d68d359\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g" Jul 06 16:48:17.129151 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:17.129151 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe227383-0435-42f2-98a8-bd027d68d359-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g\" (UID: \"fe227383-0435-42f2-98a8-bd027d68d359\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g" Jul 06 16:48:17.129326 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:17.129197 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe227383-0435-42f2-98a8-bd027d68d359-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g\" (UID: \"fe227383-0435-42f2-98a8-bd027d68d359\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g" Jul 06 16:48:17.149761 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:17.149730 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g"] Jul 06 16:48:17.230425 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:17.230392 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe227383-0435-42f2-98a8-bd027d68d359-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g\" (UID: \"fe227383-0435-42f2-98a8-bd027d68d359\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g" Jul 06 16:48:17.230604 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:17.230509 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cqfk\" (UniqueName: \"kubernetes.io/projected/fe227383-0435-42f2-98a8-bd027d68d359-kube-api-access-6cqfk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g\" (UID: \"fe227383-0435-42f2-98a8-bd027d68d359\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g" Jul 06 16:48:17.230604 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:17.230539 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe227383-0435-42f2-98a8-bd027d68d359-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g\" (UID: \"fe227383-0435-42f2-98a8-bd027d68d359\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g" Jul 06 16:48:17.230794 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:17.230773 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe227383-0435-42f2-98a8-bd027d68d359-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g\" (UID: \"fe227383-0435-42f2-98a8-bd027d68d359\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g" Jul 06 16:48:17.230874 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:17.230854 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe227383-0435-42f2-98a8-bd027d68d359-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g\" (UID: \"fe227383-0435-42f2-98a8-bd027d68d359\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g" Jul 06 16:48:17.254601 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:17.254573 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cqfk\" (UniqueName: \"kubernetes.io/projected/fe227383-0435-42f2-98a8-bd027d68d359-kube-api-access-6cqfk\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g\" (UID: \"fe227383-0435-42f2-98a8-bd027d68d359\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g" Jul 06 16:48:17.344832 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:17.344799 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g" Jul 06 16:48:17.535355 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:17.535325 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g"] Jul 06 16:48:17.537147 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:48:17.537118 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe227383_0435_42f2_98a8_bd027d68d359.slice/crio-17e24de4feae061f841efe5f53847ae03a5a9405715c87ad7300c004df95bc92 WatchSource:0}: Error finding container 17e24de4feae061f841efe5f53847ae03a5a9405715c87ad7300c004df95bc92: Status 404 returned error can't find the container with id 17e24de4feae061f841efe5f53847ae03a5a9405715c87ad7300c004df95bc92 Jul 06 16:48:17.652623 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:17.652595 2580 generic.go:356] "Generic (PLEG): container finished" podID="fe227383-0435-42f2-98a8-bd027d68d359" containerID="0aa84356b44648d6f526daa313997f702f72a7bb792e3c1f52b32ed6e9a95b85" exitCode=0 Jul 06 16:48:17.652745 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:17.652683 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g" event={"ID":"fe227383-0435-42f2-98a8-bd027d68d359","Type":"ContainerDied","Data":"0aa84356b44648d6f526daa313997f702f72a7bb792e3c1f52b32ed6e9a95b85"} Jul 06 16:48:17.652795 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:17.652738 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g" event={"ID":"fe227383-0435-42f2-98a8-bd027d68d359","Type":"ContainerStarted","Data":"17e24de4feae061f841efe5f53847ae03a5a9405715c87ad7300c004df95bc92"} Jul 06 16:48:20.150065 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:20.150032 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jsqfm"] Jul 06 16:48:20.159791 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:20.159763 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jsqfm" Jul 06 16:48:20.174972 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:20.174949 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 16:48:20.178229 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:20.178210 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-l4znf\"" Jul 06 16:48:20.179305 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:20.179287 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 16:48:20.187442 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:20.187418 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jsqfm"] Jul 06 16:48:20.253184 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:20.253153 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjgfg\" (UniqueName: \"kubernetes.io/projected/03716c94-0844-4a48-a5ef-c759c9620649-kube-api-access-tjgfg\") pod \"cert-manager-cainjector-559cdd8c87-jsqfm\" (UID: \"03716c94-0844-4a48-a5ef-c759c9620649\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jsqfm" Jul 06 16:48:20.253363 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:20.253187 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/03716c94-0844-4a48-a5ef-c759c9620649-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jsqfm\" (UID: \"03716c94-0844-4a48-a5ef-c759c9620649\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jsqfm" Jul 06 16:48:20.354433 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:20.354401 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjgfg\" (UniqueName: \"kubernetes.io/projected/03716c94-0844-4a48-a5ef-c759c9620649-kube-api-access-tjgfg\") pod \"cert-manager-cainjector-559cdd8c87-jsqfm\" (UID: \"03716c94-0844-4a48-a5ef-c759c9620649\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jsqfm" Jul 06 16:48:20.354605 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:20.354545 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/03716c94-0844-4a48-a5ef-c759c9620649-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jsqfm\" (UID: \"03716c94-0844-4a48-a5ef-c759c9620649\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jsqfm" Jul 06 16:48:20.397097 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:20.397070 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjgfg\" (UniqueName: \"kubernetes.io/projected/03716c94-0844-4a48-a5ef-c759c9620649-kube-api-access-tjgfg\") pod \"cert-manager-cainjector-559cdd8c87-jsqfm\" (UID: \"03716c94-0844-4a48-a5ef-c759c9620649\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jsqfm" Jul 06 16:48:20.398409 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:20.398383 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/03716c94-0844-4a48-a5ef-c759c9620649-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jsqfm\" (UID: \"03716c94-0844-4a48-a5ef-c759c9620649\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jsqfm" Jul 06 16:48:20.482671 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:20.482638 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jsqfm" Jul 06 16:48:20.652804 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:20.652772 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jsqfm"] Jul 06 16:48:20.655498 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:48:20.655444 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod03716c94_0844_4a48_a5ef_c759c9620649.slice/crio-14f9449fa0008e68b10a9ffbaa758eb64230d78e4da78caafe12a4bb0c407d9d WatchSource:0}: Error finding container 14f9449fa0008e68b10a9ffbaa758eb64230d78e4da78caafe12a4bb0c407d9d: Status 404 returned error can't find the container with id 14f9449fa0008e68b10a9ffbaa758eb64230d78e4da78caafe12a4bb0c407d9d Jul 06 16:48:20.666993 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:20.666966 2580 generic.go:356] "Generic (PLEG): container finished" podID="fe227383-0435-42f2-98a8-bd027d68d359" containerID="081d038d0763ffefa4d2b2ce16b5aafbc641865ea31ca99308b756b2166a25ab" exitCode=0 Jul 06 16:48:20.667101 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:20.667044 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g" event={"ID":"fe227383-0435-42f2-98a8-bd027d68d359","Type":"ContainerDied","Data":"081d038d0763ffefa4d2b2ce16b5aafbc641865ea31ca99308b756b2166a25ab"} Jul 06 16:48:20.668266 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:20.668241 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jsqfm" event={"ID":"03716c94-0844-4a48-a5ef-c759c9620649","Type":"ContainerStarted","Data":"14f9449fa0008e68b10a9ffbaa758eb64230d78e4da78caafe12a4bb0c407d9d"} Jul 06 16:48:21.675514 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:21.675422 2580 generic.go:356] "Generic (PLEG): container finished" podID="fe227383-0435-42f2-98a8-bd027d68d359" containerID="2aca2684f74e7d468cad9b3d1fea8af7f6597de38aad4340a0024d6f7aa4db36" exitCode=0 Jul 06 16:48:21.675514 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:21.675499 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g" event={"ID":"fe227383-0435-42f2-98a8-bd027d68d359","Type":"ContainerDied","Data":"2aca2684f74e7d468cad9b3d1fea8af7f6597de38aad4340a0024d6f7aa4db36"} Jul 06 16:48:22.822673 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:22.822648 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g" Jul 06 16:48:22.977660 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:22.977574 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe227383-0435-42f2-98a8-bd027d68d359-bundle\") pod \"fe227383-0435-42f2-98a8-bd027d68d359\" (UID: \"fe227383-0435-42f2-98a8-bd027d68d359\") " Jul 06 16:48:22.977660 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:22.977639 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6cqfk\" (UniqueName: \"kubernetes.io/projected/fe227383-0435-42f2-98a8-bd027d68d359-kube-api-access-6cqfk\") pod \"fe227383-0435-42f2-98a8-bd027d68d359\" (UID: \"fe227383-0435-42f2-98a8-bd027d68d359\") " Jul 06 16:48:22.977885 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:22.977685 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe227383-0435-42f2-98a8-bd027d68d359-util\") pod \"fe227383-0435-42f2-98a8-bd027d68d359\" (UID: \"fe227383-0435-42f2-98a8-bd027d68d359\") " Jul 06 16:48:22.978189 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:22.978156 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe227383-0435-42f2-98a8-bd027d68d359-bundle" (OuterVolumeSpecName: "bundle") pod "fe227383-0435-42f2-98a8-bd027d68d359" (UID: "fe227383-0435-42f2-98a8-bd027d68d359"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:48:22.979989 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:22.979962 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe227383-0435-42f2-98a8-bd027d68d359-kube-api-access-6cqfk" (OuterVolumeSpecName: "kube-api-access-6cqfk") pod "fe227383-0435-42f2-98a8-bd027d68d359" (UID: "fe227383-0435-42f2-98a8-bd027d68d359"). InnerVolumeSpecName "kube-api-access-6cqfk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:48:22.982734 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:22.982702 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe227383-0435-42f2-98a8-bd027d68d359-util" (OuterVolumeSpecName: "util") pod "fe227383-0435-42f2-98a8-bd027d68d359" (UID: "fe227383-0435-42f2-98a8-bd027d68d359"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:48:23.078684 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:23.078647 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe227383-0435-42f2-98a8-bd027d68d359-util\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:48:23.078684 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:23.078683 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe227383-0435-42f2-98a8-bd027d68d359-bundle\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:48:23.078880 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:23.078699 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6cqfk\" (UniqueName: \"kubernetes.io/projected/fe227383-0435-42f2-98a8-bd027d68d359-kube-api-access-6cqfk\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:48:23.688030 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:23.688002 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g" Jul 06 16:48:23.688223 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:23.688025 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9gw4g" event={"ID":"fe227383-0435-42f2-98a8-bd027d68d359","Type":"ContainerDied","Data":"17e24de4feae061f841efe5f53847ae03a5a9405715c87ad7300c004df95bc92"} Jul 06 16:48:23.688223 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:23.688072 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17e24de4feae061f841efe5f53847ae03a5a9405715c87ad7300c004df95bc92" Jul 06 16:48:24.693461 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:24.693425 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jsqfm" event={"ID":"03716c94-0844-4a48-a5ef-c759c9620649","Type":"ContainerStarted","Data":"57dcc48b2cd28ed80a82e6414af0d93211440c0f05fea2fc3f53ba1ba0d0541c"} Jul 06 16:48:24.736354 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:24.736303 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jsqfm" podStartSLOduration=1.513867515 podStartE2EDuration="4.736286289s" totalImagesPullingTime="3.222418774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:48:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:48:24.735131097 +0000 UTC m=+618.912990214" watchObservedRunningTime="2026-07-06 16:48:24.736286289 +0000 UTC m=+618.914145400" Jul 06 16:48:29.145526 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.145408 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lnwhq"] Jul 06 16:48:29.145979 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.145843 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe227383-0435-42f2-98a8-bd027d68d359" containerName="util" Jul 06 16:48:29.145979 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.145862 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe227383-0435-42f2-98a8-bd027d68d359" containerName="util" Jul 06 16:48:29.145979 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.145884 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe227383-0435-42f2-98a8-bd027d68d359" containerName="pull" Jul 06 16:48:29.145979 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.145891 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe227383-0435-42f2-98a8-bd027d68d359" containerName="pull" Jul 06 16:48:29.145979 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.145904 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe227383-0435-42f2-98a8-bd027d68d359" containerName="extract" Jul 06 16:48:29.145979 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.145912 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe227383-0435-42f2-98a8-bd027d68d359" containerName="extract" Jul 06 16:48:29.146256 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.145995 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe227383-0435-42f2-98a8-bd027d68d359" containerName="extract" Jul 06 16:48:29.149686 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.149665 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lnwhq" Jul 06 16:48:29.152460 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.152440 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 16:48:29.152593 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.152514 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-rl9kn\"" Jul 06 16:48:29.152905 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.152887 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 16:48:29.161109 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.161089 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lnwhq"] Jul 06 16:48:29.328722 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.328682 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbdb8a1e-75d3-40ba-94a9-36f63fe4788c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-lnwhq\" (UID: \"bbdb8a1e-75d3-40ba-94a9-36f63fe4788c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lnwhq" Jul 06 16:48:29.328722 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.328730 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g2zj\" (UniqueName: \"kubernetes.io/projected/bbdb8a1e-75d3-40ba-94a9-36f63fe4788c-kube-api-access-2g2zj\") pod \"openshift-lws-operator-fd8ccff4c-lnwhq\" (UID: \"bbdb8a1e-75d3-40ba-94a9-36f63fe4788c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lnwhq" Jul 06 16:48:29.429879 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.429784 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbdb8a1e-75d3-40ba-94a9-36f63fe4788c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-lnwhq\" (UID: \"bbdb8a1e-75d3-40ba-94a9-36f63fe4788c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lnwhq" Jul 06 16:48:29.429879 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.429837 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g2zj\" (UniqueName: \"kubernetes.io/projected/bbdb8a1e-75d3-40ba-94a9-36f63fe4788c-kube-api-access-2g2zj\") pod \"openshift-lws-operator-fd8ccff4c-lnwhq\" (UID: \"bbdb8a1e-75d3-40ba-94a9-36f63fe4788c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lnwhq" Jul 06 16:48:29.430253 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.430231 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbdb8a1e-75d3-40ba-94a9-36f63fe4788c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-lnwhq\" (UID: \"bbdb8a1e-75d3-40ba-94a9-36f63fe4788c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lnwhq" Jul 06 16:48:29.439912 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.439887 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g2zj\" (UniqueName: \"kubernetes.io/projected/bbdb8a1e-75d3-40ba-94a9-36f63fe4788c-kube-api-access-2g2zj\") pod \"openshift-lws-operator-fd8ccff4c-lnwhq\" (UID: \"bbdb8a1e-75d3-40ba-94a9-36f63fe4788c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lnwhq" Jul 06 16:48:29.458854 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.458833 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lnwhq" Jul 06 16:48:29.589495 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.589444 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lnwhq"] Jul 06 16:48:29.591352 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:48:29.591327 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbbdb8a1e_75d3_40ba_94a9_36f63fe4788c.slice/crio-35c55645e9e9658b73441f0075c1e07e5eb5088f6f8deedf20d656f0f490987b WatchSource:0}: Error finding container 35c55645e9e9658b73441f0075c1e07e5eb5088f6f8deedf20d656f0f490987b: Status 404 returned error can't find the container with id 35c55645e9e9658b73441f0075c1e07e5eb5088f6f8deedf20d656f0f490987b Jul 06 16:48:29.719734 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:29.719667 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lnwhq" event={"ID":"bbdb8a1e-75d3-40ba-94a9-36f63fe4788c","Type":"ContainerStarted","Data":"35c55645e9e9658b73441f0075c1e07e5eb5088f6f8deedf20d656f0f490987b"} Jul 06 16:48:31.729878 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:31.729796 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lnwhq" event={"ID":"bbdb8a1e-75d3-40ba-94a9-36f63fe4788c","Type":"ContainerStarted","Data":"0adfc5d83cdda8d844abc7b52c92afa9119a74c42261a2c6f1e6b0628b68e859"} Jul 06 16:48:31.755770 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:31.755722 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-lnwhq" podStartSLOduration=0.900720217 podStartE2EDuration="2.755708567s" totalImagesPullingTime="1.85498835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:48:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:48:31.755045195 +0000 UTC m=+625.932904324" watchObservedRunningTime="2026-07-06 16:48:31.755708567 +0000 UTC m=+625.933567679" Jul 06 16:48:34.019525 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:34.019483 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v"] Jul 06 16:48:34.038185 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:34.038147 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v"] Jul 06 16:48:34.038326 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:34.038276 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v" Jul 06 16:48:34.041238 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:34.041217 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 16:48:34.041350 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:34.041216 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 16:48:34.041741 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:34.041726 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mxvhm\"" Jul 06 16:48:34.167526 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:34.167487 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf5xt\" (UniqueName: \"kubernetes.io/projected/3d7f15d9-08fa-4c0f-8375-65817a24d215-kube-api-access-xf5xt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v\" (UID: \"3d7f15d9-08fa-4c0f-8375-65817a24d215\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v" Jul 06 16:48:34.167715 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:34.167535 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d7f15d9-08fa-4c0f-8375-65817a24d215-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v\" (UID: \"3d7f15d9-08fa-4c0f-8375-65817a24d215\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v" Jul 06 16:48:34.167715 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:34.167606 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d7f15d9-08fa-4c0f-8375-65817a24d215-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v\" (UID: \"3d7f15d9-08fa-4c0f-8375-65817a24d215\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v" Jul 06 16:48:34.269061 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:34.269027 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf5xt\" (UniqueName: \"kubernetes.io/projected/3d7f15d9-08fa-4c0f-8375-65817a24d215-kube-api-access-xf5xt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v\" (UID: \"3d7f15d9-08fa-4c0f-8375-65817a24d215\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v" Jul 06 16:48:34.269061 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:34.269064 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d7f15d9-08fa-4c0f-8375-65817a24d215-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v\" (UID: \"3d7f15d9-08fa-4c0f-8375-65817a24d215\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v" Jul 06 16:48:34.269267 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:34.269101 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d7f15d9-08fa-4c0f-8375-65817a24d215-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v\" (UID: \"3d7f15d9-08fa-4c0f-8375-65817a24d215\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v" Jul 06 16:48:34.269426 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:34.269406 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d7f15d9-08fa-4c0f-8375-65817a24d215-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v\" (UID: \"3d7f15d9-08fa-4c0f-8375-65817a24d215\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v" Jul 06 16:48:34.269464 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:34.269445 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d7f15d9-08fa-4c0f-8375-65817a24d215-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v\" (UID: \"3d7f15d9-08fa-4c0f-8375-65817a24d215\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v" Jul 06 16:48:34.280305 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:34.280262 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf5xt\" (UniqueName: \"kubernetes.io/projected/3d7f15d9-08fa-4c0f-8375-65817a24d215-kube-api-access-xf5xt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v\" (UID: \"3d7f15d9-08fa-4c0f-8375-65817a24d215\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v" Jul 06 16:48:34.347748 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:34.347719 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v" Jul 06 16:48:34.503688 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:34.503662 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v"] Jul 06 16:48:34.505372 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:48:34.505342 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d7f15d9_08fa_4c0f_8375_65817a24d215.slice/crio-ab60bff5d6f57030acb2a6677ad745fd4493916954710a04f61592c3737b3688 WatchSource:0}: Error finding container ab60bff5d6f57030acb2a6677ad745fd4493916954710a04f61592c3737b3688: Status 404 returned error can't find the container with id ab60bff5d6f57030acb2a6677ad745fd4493916954710a04f61592c3737b3688 Jul 06 16:48:34.744544 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:34.744512 2580 generic.go:356] "Generic (PLEG): container finished" podID="3d7f15d9-08fa-4c0f-8375-65817a24d215" containerID="32292f9eee6b25d1c3607c892b4cd4b89ec9b7d322ccae1ed61dc35d2ee23292" exitCode=0 Jul 06 16:48:34.744728 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:34.744605 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v" event={"ID":"3d7f15d9-08fa-4c0f-8375-65817a24d215","Type":"ContainerDied","Data":"32292f9eee6b25d1c3607c892b4cd4b89ec9b7d322ccae1ed61dc35d2ee23292"} Jul 06 16:48:34.744728 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:34.744653 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v" event={"ID":"3d7f15d9-08fa-4c0f-8375-65817a24d215","Type":"ContainerStarted","Data":"ab60bff5d6f57030acb2a6677ad745fd4493916954710a04f61592c3737b3688"} Jul 06 16:48:36.755130 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:36.755087 2580 generic.go:356] "Generic (PLEG): container finished" podID="3d7f15d9-08fa-4c0f-8375-65817a24d215" containerID="d0c6b7493ab0e5002edaa14836eb434b50343f2687f7a05daf1bc462ba9b91a7" exitCode=0 Jul 06 16:48:36.755545 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:36.755152 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v" event={"ID":"3d7f15d9-08fa-4c0f-8375-65817a24d215","Type":"ContainerDied","Data":"d0c6b7493ab0e5002edaa14836eb434b50343f2687f7a05daf1bc462ba9b91a7"} Jul 06 16:48:37.761637 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:37.761605 2580 generic.go:356] "Generic (PLEG): container finished" podID="3d7f15d9-08fa-4c0f-8375-65817a24d215" containerID="a67e914fda52051beb00ca1def8fc4caa616a35a5ec15042940df26744adc9c8" exitCode=0 Jul 06 16:48:37.762025 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:37.761686 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v" event={"ID":"3d7f15d9-08fa-4c0f-8375-65817a24d215","Type":"ContainerDied","Data":"a67e914fda52051beb00ca1def8fc4caa616a35a5ec15042940df26744adc9c8"} Jul 06 16:48:38.895307 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:38.895282 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v" Jul 06 16:48:39.007340 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:39.007308 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xf5xt\" (UniqueName: \"kubernetes.io/projected/3d7f15d9-08fa-4c0f-8375-65817a24d215-kube-api-access-xf5xt\") pod \"3d7f15d9-08fa-4c0f-8375-65817a24d215\" (UID: \"3d7f15d9-08fa-4c0f-8375-65817a24d215\") " Jul 06 16:48:39.007340 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:39.007342 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d7f15d9-08fa-4c0f-8375-65817a24d215-util\") pod \"3d7f15d9-08fa-4c0f-8375-65817a24d215\" (UID: \"3d7f15d9-08fa-4c0f-8375-65817a24d215\") " Jul 06 16:48:39.007559 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:39.007425 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d7f15d9-08fa-4c0f-8375-65817a24d215-bundle\") pod \"3d7f15d9-08fa-4c0f-8375-65817a24d215\" (UID: \"3d7f15d9-08fa-4c0f-8375-65817a24d215\") " Jul 06 16:48:39.008164 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:39.008133 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d7f15d9-08fa-4c0f-8375-65817a24d215-bundle" (OuterVolumeSpecName: "bundle") pod "3d7f15d9-08fa-4c0f-8375-65817a24d215" (UID: "3d7f15d9-08fa-4c0f-8375-65817a24d215"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:48:39.009386 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:39.009361 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d7f15d9-08fa-4c0f-8375-65817a24d215-kube-api-access-xf5xt" (OuterVolumeSpecName: "kube-api-access-xf5xt") pod "3d7f15d9-08fa-4c0f-8375-65817a24d215" (UID: "3d7f15d9-08fa-4c0f-8375-65817a24d215"). InnerVolumeSpecName "kube-api-access-xf5xt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:48:39.012885 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:39.012865 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d7f15d9-08fa-4c0f-8375-65817a24d215-util" (OuterVolumeSpecName: "util") pod "3d7f15d9-08fa-4c0f-8375-65817a24d215" (UID: "3d7f15d9-08fa-4c0f-8375-65817a24d215"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:48:39.108533 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:39.108496 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d7f15d9-08fa-4c0f-8375-65817a24d215-bundle\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:48:39.108533 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:39.108531 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xf5xt\" (UniqueName: \"kubernetes.io/projected/3d7f15d9-08fa-4c0f-8375-65817a24d215-kube-api-access-xf5xt\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:48:39.108533 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:39.108542 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d7f15d9-08fa-4c0f-8375-65817a24d215-util\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:48:39.772281 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:39.772241 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v" event={"ID":"3d7f15d9-08fa-4c0f-8375-65817a24d215","Type":"ContainerDied","Data":"ab60bff5d6f57030acb2a6677ad745fd4493916954710a04f61592c3737b3688"} Jul 06 16:48:39.772456 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:39.772300 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab60bff5d6f57030acb2a6677ad745fd4493916954710a04f61592c3737b3688" Jul 06 16:48:39.772456 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:39.772254 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af25p7v" Jul 06 16:48:51.704636 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.704603 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j"] Jul 06 16:48:51.705098 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.704948 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d7f15d9-08fa-4c0f-8375-65817a24d215" containerName="util" Jul 06 16:48:51.705098 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.704960 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d7f15d9-08fa-4c0f-8375-65817a24d215" containerName="util" Jul 06 16:48:51.705098 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.704977 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d7f15d9-08fa-4c0f-8375-65817a24d215" containerName="pull" Jul 06 16:48:51.705098 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.704982 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d7f15d9-08fa-4c0f-8375-65817a24d215" containerName="pull" Jul 06 16:48:51.705098 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.704988 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d7f15d9-08fa-4c0f-8375-65817a24d215" containerName="extract" Jul 06 16:48:51.705098 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.704994 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d7f15d9-08fa-4c0f-8375-65817a24d215" containerName="extract" Jul 06 16:48:51.705098 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.705045 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d7f15d9-08fa-4c0f-8375-65817a24d215" containerName="extract" Jul 06 16:48:51.708194 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.708175 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" Jul 06 16:48:51.711525 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.711500 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kwj6w\"" Jul 06 16:48:51.711671 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.711653 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 16:48:51.711753 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.711678 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 16:48:51.712027 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.712015 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 16:48:51.712076 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.712031 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 16:48:51.729326 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.729299 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j"] Jul 06 16:48:51.796077 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.796047 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/245f6ba2-b372-4443-895c-4237ac62053a-manifests\") pod \"opendatahub-operator-controller-manager-d66bcd77d-4pr8j\" (UID: \"245f6ba2-b372-4443-895c-4237ac62053a\") " pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" Jul 06 16:48:51.796191 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.796091 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fc8r\" (UniqueName: \"kubernetes.io/projected/245f6ba2-b372-4443-895c-4237ac62053a-kube-api-access-5fc8r\") pod \"opendatahub-operator-controller-manager-d66bcd77d-4pr8j\" (UID: \"245f6ba2-b372-4443-895c-4237ac62053a\") " pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" Jul 06 16:48:51.796191 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.796178 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/245f6ba2-b372-4443-895c-4237ac62053a-tmp\") pod \"opendatahub-operator-controller-manager-d66bcd77d-4pr8j\" (UID: \"245f6ba2-b372-4443-895c-4237ac62053a\") " pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" Jul 06 16:48:51.796267 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.796221 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/245f6ba2-b372-4443-895c-4237ac62053a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d66bcd77d-4pr8j\" (UID: \"245f6ba2-b372-4443-895c-4237ac62053a\") " pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" Jul 06 16:48:51.796267 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.796250 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/245f6ba2-b372-4443-895c-4237ac62053a-webhook-cert\") pod \"opendatahub-operator-controller-manager-d66bcd77d-4pr8j\" (UID: \"245f6ba2-b372-4443-895c-4237ac62053a\") " pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" Jul 06 16:48:51.897395 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.897354 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/245f6ba2-b372-4443-895c-4237ac62053a-tmp\") pod \"opendatahub-operator-controller-manager-d66bcd77d-4pr8j\" (UID: \"245f6ba2-b372-4443-895c-4237ac62053a\") " pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" Jul 06 16:48:51.897548 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.897421 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/245f6ba2-b372-4443-895c-4237ac62053a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d66bcd77d-4pr8j\" (UID: \"245f6ba2-b372-4443-895c-4237ac62053a\") " pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" Jul 06 16:48:51.897548 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.897461 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/245f6ba2-b372-4443-895c-4237ac62053a-webhook-cert\") pod \"opendatahub-operator-controller-manager-d66bcd77d-4pr8j\" (UID: \"245f6ba2-b372-4443-895c-4237ac62053a\") " pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" Jul 06 16:48:51.897548 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.897516 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/245f6ba2-b372-4443-895c-4237ac62053a-manifests\") pod \"opendatahub-operator-controller-manager-d66bcd77d-4pr8j\" (UID: \"245f6ba2-b372-4443-895c-4237ac62053a\") " pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" Jul 06 16:48:51.897707 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.897558 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fc8r\" (UniqueName: \"kubernetes.io/projected/245f6ba2-b372-4443-895c-4237ac62053a-kube-api-access-5fc8r\") pod \"opendatahub-operator-controller-manager-d66bcd77d-4pr8j\" (UID: \"245f6ba2-b372-4443-895c-4237ac62053a\") " pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" Jul 06 16:48:51.897908 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.897886 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/245f6ba2-b372-4443-895c-4237ac62053a-manifests\") pod \"opendatahub-operator-controller-manager-d66bcd77d-4pr8j\" (UID: \"245f6ba2-b372-4443-895c-4237ac62053a\") " pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" Jul 06 16:48:51.899924 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.899896 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/245f6ba2-b372-4443-895c-4237ac62053a-tmp\") pod \"opendatahub-operator-controller-manager-d66bcd77d-4pr8j\" (UID: \"245f6ba2-b372-4443-895c-4237ac62053a\") " pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" Jul 06 16:48:51.900006 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.899948 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/245f6ba2-b372-4443-895c-4237ac62053a-webhook-cert\") pod \"opendatahub-operator-controller-manager-d66bcd77d-4pr8j\" (UID: \"245f6ba2-b372-4443-895c-4237ac62053a\") " pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" Jul 06 16:48:51.900081 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.900065 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/245f6ba2-b372-4443-895c-4237ac62053a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d66bcd77d-4pr8j\" (UID: \"245f6ba2-b372-4443-895c-4237ac62053a\") " pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" Jul 06 16:48:51.907045 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:51.907021 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fc8r\" (UniqueName: \"kubernetes.io/projected/245f6ba2-b372-4443-895c-4237ac62053a-kube-api-access-5fc8r\") pod \"opendatahub-operator-controller-manager-d66bcd77d-4pr8j\" (UID: \"245f6ba2-b372-4443-895c-4237ac62053a\") " pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" Jul 06 16:48:52.018406 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:52.018328 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" Jul 06 16:48:52.157265 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:52.157240 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j"] Jul 06 16:48:52.158928 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:48:52.158902 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod245f6ba2_b372_4443_895c_4237ac62053a.slice/crio-3b8adbf8775d6d79631fab672f51a6e7ce9b8919fb04e84ea3906c4d3750a4ae WatchSource:0}: Error finding container 3b8adbf8775d6d79631fab672f51a6e7ce9b8919fb04e84ea3906c4d3750a4ae: Status 404 returned error can't find the container with id 3b8adbf8775d6d79631fab672f51a6e7ce9b8919fb04e84ea3906c4d3750a4ae Jul 06 16:48:52.832439 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:52.832398 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" event={"ID":"245f6ba2-b372-4443-895c-4237ac62053a","Type":"ContainerStarted","Data":"3b8adbf8775d6d79631fab672f51a6e7ce9b8919fb04e84ea3906c4d3750a4ae"} Jul 06 16:48:55.849016 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:55.848983 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" event={"ID":"245f6ba2-b372-4443-895c-4237ac62053a","Type":"ContainerStarted","Data":"22a11e7b1ce5e887ae2d7e3c06c702122477ef46a0ac5f0915ab63f2ba258ad4"} Jul 06 16:48:56.855070 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:56.855042 2580 generic.go:356] "Generic (PLEG): container finished" podID="245f6ba2-b372-4443-895c-4237ac62053a" containerID="22a11e7b1ce5e887ae2d7e3c06c702122477ef46a0ac5f0915ab63f2ba258ad4" exitCode=0 Jul 06 16:48:56.855384 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:56.855135 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" event={"ID":"245f6ba2-b372-4443-895c-4237ac62053a","Type":"ContainerDied","Data":"22a11e7b1ce5e887ae2d7e3c06c702122477ef46a0ac5f0915ab63f2ba258ad4"} Jul 06 16:48:57.861678 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:57.861642 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" event={"ID":"245f6ba2-b372-4443-895c-4237ac62053a","Type":"ContainerStarted","Data":"4429c2f7ec9bda9104fc748c61bba9b819b3531a0af438086e32a56b45f218fd"} Jul 06 16:48:57.862174 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:57.861791 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" Jul 06 16:48:57.971076 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:48:57.970899 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" podStartSLOduration=1.9681846410000001 podStartE2EDuration="6.970884261s" totalImagesPullingTime="3.927574155s" totalInitContainerRuntime="1.075125465s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:48:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:48:57.966241729 +0000 UTC m=+652.144100840" watchObservedRunningTime="2026-07-06 16:48:57.970884261 +0000 UTC m=+652.148743372" Jul 06 16:49:08.868059 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:08.868023 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d66bcd77d-4pr8j" Jul 06 16:49:11.463252 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:11.463221 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5548f7d4db-lcsdz"] Jul 06 16:49:11.472043 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:11.472022 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5548f7d4db-lcsdz" Jul 06 16:49:11.474514 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:11.474494 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 16:49:11.474816 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:11.474800 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-pxdww\"" Jul 06 16:49:11.474907 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:11.474805 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 16:49:11.491205 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:11.491181 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5548f7d4db-lcsdz"] Jul 06 16:49:11.551314 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:11.551287 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/152039b3-78fb-464d-8915-50995f2c6b35-tmp\") pod \"kube-auth-proxy-5548f7d4db-lcsdz\" (UID: \"152039b3-78fb-464d-8915-50995f2c6b35\") " pod="openshift-ingress/kube-auth-proxy-5548f7d4db-lcsdz" Jul 06 16:49:11.551512 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:11.551335 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqbsl\" (UniqueName: \"kubernetes.io/projected/152039b3-78fb-464d-8915-50995f2c6b35-kube-api-access-gqbsl\") pod \"kube-auth-proxy-5548f7d4db-lcsdz\" (UID: \"152039b3-78fb-464d-8915-50995f2c6b35\") " pod="openshift-ingress/kube-auth-proxy-5548f7d4db-lcsdz" Jul 06 16:49:11.551512 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:11.551433 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/152039b3-78fb-464d-8915-50995f2c6b35-tls-certs\") pod \"kube-auth-proxy-5548f7d4db-lcsdz\" (UID: \"152039b3-78fb-464d-8915-50995f2c6b35\") " pod="openshift-ingress/kube-auth-proxy-5548f7d4db-lcsdz" Jul 06 16:49:11.652802 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:11.652756 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/152039b3-78fb-464d-8915-50995f2c6b35-tmp\") pod \"kube-auth-proxy-5548f7d4db-lcsdz\" (UID: \"152039b3-78fb-464d-8915-50995f2c6b35\") " pod="openshift-ingress/kube-auth-proxy-5548f7d4db-lcsdz" Jul 06 16:49:11.653000 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:11.652828 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqbsl\" (UniqueName: \"kubernetes.io/projected/152039b3-78fb-464d-8915-50995f2c6b35-kube-api-access-gqbsl\") pod \"kube-auth-proxy-5548f7d4db-lcsdz\" (UID: \"152039b3-78fb-464d-8915-50995f2c6b35\") " pod="openshift-ingress/kube-auth-proxy-5548f7d4db-lcsdz" Jul 06 16:49:11.653000 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:11.652887 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/152039b3-78fb-464d-8915-50995f2c6b35-tls-certs\") pod \"kube-auth-proxy-5548f7d4db-lcsdz\" (UID: \"152039b3-78fb-464d-8915-50995f2c6b35\") " pod="openshift-ingress/kube-auth-proxy-5548f7d4db-lcsdz" Jul 06 16:49:11.655561 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:11.655500 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/152039b3-78fb-464d-8915-50995f2c6b35-tmp\") pod \"kube-auth-proxy-5548f7d4db-lcsdz\" (UID: \"152039b3-78fb-464d-8915-50995f2c6b35\") " pod="openshift-ingress/kube-auth-proxy-5548f7d4db-lcsdz" Jul 06 16:49:11.655884 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:11.655866 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/152039b3-78fb-464d-8915-50995f2c6b35-tls-certs\") pod \"kube-auth-proxy-5548f7d4db-lcsdz\" (UID: \"152039b3-78fb-464d-8915-50995f2c6b35\") " pod="openshift-ingress/kube-auth-proxy-5548f7d4db-lcsdz" Jul 06 16:49:11.671369 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:11.671338 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqbsl\" (UniqueName: \"kubernetes.io/projected/152039b3-78fb-464d-8915-50995f2c6b35-kube-api-access-gqbsl\") pod \"kube-auth-proxy-5548f7d4db-lcsdz\" (UID: \"152039b3-78fb-464d-8915-50995f2c6b35\") " pod="openshift-ingress/kube-auth-proxy-5548f7d4db-lcsdz" Jul 06 16:49:11.782892 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:11.782807 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5548f7d4db-lcsdz" Jul 06 16:49:11.930464 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:11.930434 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5548f7d4db-lcsdz"] Jul 06 16:49:11.933453 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:49:11.933406 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod152039b3_78fb_464d_8915_50995f2c6b35.slice/crio-8f624e4e8ae29eda741f1bd03897c52db55a10226f24b69d3e03f01161badc3e WatchSource:0}: Error finding container 8f624e4e8ae29eda741f1bd03897c52db55a10226f24b69d3e03f01161badc3e: Status 404 returned error can't find the container with id 8f624e4e8ae29eda741f1bd03897c52db55a10226f24b69d3e03f01161badc3e Jul 06 16:49:12.031096 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:12.031064 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-576h9"] Jul 06 16:49:12.035657 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:12.035596 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-576h9" Jul 06 16:49:12.038299 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:12.038273 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-22fh6\"" Jul 06 16:49:12.038411 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:12.038316 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 06 16:49:12.060055 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:12.060022 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-576h9"] Jul 06 16:49:12.158025 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:12.157974 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdnb9\" (UniqueName: \"kubernetes.io/projected/fc68b1a1-028e-4d77-a084-c17a8757bc79-kube-api-access-fdnb9\") pod \"odh-model-controller-7998c67dc4-576h9\" (UID: \"fc68b1a1-028e-4d77-a084-c17a8757bc79\") " pod="opendatahub/odh-model-controller-7998c67dc4-576h9" Jul 06 16:49:12.158194 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:12.158048 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc68b1a1-028e-4d77-a084-c17a8757bc79-cert\") pod \"odh-model-controller-7998c67dc4-576h9\" (UID: \"fc68b1a1-028e-4d77-a084-c17a8757bc79\") " pod="opendatahub/odh-model-controller-7998c67dc4-576h9" Jul 06 16:49:12.259209 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:12.259170 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdnb9\" (UniqueName: \"kubernetes.io/projected/fc68b1a1-028e-4d77-a084-c17a8757bc79-kube-api-access-fdnb9\") pod \"odh-model-controller-7998c67dc4-576h9\" (UID: \"fc68b1a1-028e-4d77-a084-c17a8757bc79\") " pod="opendatahub/odh-model-controller-7998c67dc4-576h9" Jul 06 16:49:12.259389 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:12.259239 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc68b1a1-028e-4d77-a084-c17a8757bc79-cert\") pod \"odh-model-controller-7998c67dc4-576h9\" (UID: \"fc68b1a1-028e-4d77-a084-c17a8757bc79\") " pod="opendatahub/odh-model-controller-7998c67dc4-576h9" Jul 06 16:49:12.259389 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:49:12.259372 2580 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 06 16:49:12.259535 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:49:12.259449 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc68b1a1-028e-4d77-a084-c17a8757bc79-cert podName:fc68b1a1-028e-4d77-a084-c17a8757bc79 nodeName:}" failed. No retries permitted until 2026-07-06 16:49:12.759426752 +0000 UTC m=+666.937285850 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/fc68b1a1-028e-4d77-a084-c17a8757bc79-cert") pod "odh-model-controller-7998c67dc4-576h9" (UID: "fc68b1a1-028e-4d77-a084-c17a8757bc79") : secret "odh-model-controller-webhook-cert" not found Jul 06 16:49:12.274435 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:12.274403 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdnb9\" (UniqueName: \"kubernetes.io/projected/fc68b1a1-028e-4d77-a084-c17a8757bc79-kube-api-access-fdnb9\") pod \"odh-model-controller-7998c67dc4-576h9\" (UID: \"fc68b1a1-028e-4d77-a084-c17a8757bc79\") " pod="opendatahub/odh-model-controller-7998c67dc4-576h9" Jul 06 16:49:12.764666 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:12.764627 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc68b1a1-028e-4d77-a084-c17a8757bc79-cert\") pod \"odh-model-controller-7998c67dc4-576h9\" (UID: \"fc68b1a1-028e-4d77-a084-c17a8757bc79\") " pod="opendatahub/odh-model-controller-7998c67dc4-576h9" Jul 06 16:49:12.767114 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:12.767083 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fc68b1a1-028e-4d77-a084-c17a8757bc79-cert\") pod \"odh-model-controller-7998c67dc4-576h9\" (UID: \"fc68b1a1-028e-4d77-a084-c17a8757bc79\") " pod="opendatahub/odh-model-controller-7998c67dc4-576h9" Jul 06 16:49:12.933813 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:12.933747 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5548f7d4db-lcsdz" event={"ID":"152039b3-78fb-464d-8915-50995f2c6b35","Type":"ContainerStarted","Data":"8f624e4e8ae29eda741f1bd03897c52db55a10226f24b69d3e03f01161badc3e"} Jul 06 16:49:12.946426 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:12.946266 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-576h9" Jul 06 16:49:13.127562 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:13.127485 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-576h9"] Jul 06 16:49:13.130247 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:49:13.130213 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc68b1a1_028e_4d77_a084_c17a8757bc79.slice/crio-347f02fc2dbd0440db2094c92257e7c70e3bfb46175d4158baa24f545a9902c1 WatchSource:0}: Error finding container 347f02fc2dbd0440db2094c92257e7c70e3bfb46175d4158baa24f545a9902c1: Status 404 returned error can't find the container with id 347f02fc2dbd0440db2094c92257e7c70e3bfb46175d4158baa24f545a9902c1 Jul 06 16:49:13.520157 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:13.520123 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-blccs"] Jul 06 16:49:13.524617 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:13.524601 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-blccs" Jul 06 16:49:13.526591 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:13.526570 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 06 16:49:13.526686 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:13.526623 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-jvz96\"" Jul 06 16:49:13.570840 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:13.570789 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1f5e726-3704-461f-a075-7a2a791f1dfc-tls-certs\") pod \"model-serving-api-54d89bd5fc-blccs\" (UID: \"e1f5e726-3704-461f-a075-7a2a791f1dfc\") " pod="opendatahub/model-serving-api-54d89bd5fc-blccs" Jul 06 16:49:13.570840 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:13.570839 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rszt\" (UniqueName: \"kubernetes.io/projected/e1f5e726-3704-461f-a075-7a2a791f1dfc-kube-api-access-2rszt\") pod \"model-serving-api-54d89bd5fc-blccs\" (UID: \"e1f5e726-3704-461f-a075-7a2a791f1dfc\") " pod="opendatahub/model-serving-api-54d89bd5fc-blccs" Jul 06 16:49:13.590929 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:13.590901 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-blccs"] Jul 06 16:49:13.671659 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:13.671626 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1f5e726-3704-461f-a075-7a2a791f1dfc-tls-certs\") pod \"model-serving-api-54d89bd5fc-blccs\" (UID: \"e1f5e726-3704-461f-a075-7a2a791f1dfc\") " pod="opendatahub/model-serving-api-54d89bd5fc-blccs" Jul 06 16:49:13.671844 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:13.671674 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rszt\" (UniqueName: \"kubernetes.io/projected/e1f5e726-3704-461f-a075-7a2a791f1dfc-kube-api-access-2rszt\") pod \"model-serving-api-54d89bd5fc-blccs\" (UID: \"e1f5e726-3704-461f-a075-7a2a791f1dfc\") " pod="opendatahub/model-serving-api-54d89bd5fc-blccs" Jul 06 16:49:13.674120 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:13.674097 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1f5e726-3704-461f-a075-7a2a791f1dfc-tls-certs\") pod \"model-serving-api-54d89bd5fc-blccs\" (UID: \"e1f5e726-3704-461f-a075-7a2a791f1dfc\") " pod="opendatahub/model-serving-api-54d89bd5fc-blccs" Jul 06 16:49:13.682318 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:13.682287 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rszt\" (UniqueName: \"kubernetes.io/projected/e1f5e726-3704-461f-a075-7a2a791f1dfc-kube-api-access-2rszt\") pod \"model-serving-api-54d89bd5fc-blccs\" (UID: \"e1f5e726-3704-461f-a075-7a2a791f1dfc\") " pod="opendatahub/model-serving-api-54d89bd5fc-blccs" Jul 06 16:49:13.834845 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:13.834810 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-blccs" Jul 06 16:49:13.943084 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:13.942625 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-576h9" event={"ID":"fc68b1a1-028e-4d77-a084-c17a8757bc79","Type":"ContainerStarted","Data":"347f02fc2dbd0440db2094c92257e7c70e3bfb46175d4158baa24f545a9902c1"} Jul 06 16:49:14.017532 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:14.017502 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-blccs"] Jul 06 16:49:14.019986 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:49:14.019956 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1f5e726_3704_461f_a075_7a2a791f1dfc.slice/crio-4b8395eaf94ae5a1855876cd761755c88b30b19e82630429e5bf7657dbd8e9b0 WatchSource:0}: Error finding container 4b8395eaf94ae5a1855876cd761755c88b30b19e82630429e5bf7657dbd8e9b0: Status 404 returned error can't find the container with id 4b8395eaf94ae5a1855876cd761755c88b30b19e82630429e5bf7657dbd8e9b0 Jul 06 16:49:14.951845 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:14.951798 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-blccs" event={"ID":"e1f5e726-3704-461f-a075-7a2a791f1dfc","Type":"ContainerStarted","Data":"4b8395eaf94ae5a1855876cd761755c88b30b19e82630429e5bf7657dbd8e9b0"} Jul 06 16:49:17.970189 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:17.970149 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-blccs" event={"ID":"e1f5e726-3704-461f-a075-7a2a791f1dfc","Type":"ContainerStarted","Data":"d6b4dc956dc7448cf1eaabe5882cf32ad9808797547af953c4bee36e4911cfbb"} Jul 06 16:49:17.970712 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:17.970429 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-blccs" Jul 06 16:49:17.972146 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:17.972095 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-576h9" event={"ID":"fc68b1a1-028e-4d77-a084-c17a8757bc79","Type":"ContainerStarted","Data":"58fa5154def5c118085fea8343118a5704e08819a9f7934a16c93a289de53513"} Jul 06 16:49:17.972288 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:17.972246 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-576h9" Jul 06 16:49:17.973819 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:17.973796 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5548f7d4db-lcsdz" event={"ID":"152039b3-78fb-464d-8915-50995f2c6b35","Type":"ContainerStarted","Data":"a1e252dac5d011b9a88cce25cd544bee920818d491241d12a8a10ef2ac1ff939"} Jul 06 16:49:17.991342 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:17.991295 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-blccs" podStartSLOduration=1.4828286880000001 podStartE2EDuration="4.99127912s" totalImagesPullingTime="3.508450432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:49:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:49:17.990932202 +0000 UTC m=+672.168791314" watchObservedRunningTime="2026-07-06 16:49:17.99127912 +0000 UTC m=+672.169138232" Jul 06 16:49:18.014322 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:18.014280 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-576h9" podStartSLOduration=2.59813388 podStartE2EDuration="7.01426786s" totalImagesPullingTime="4.41613398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:49:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:49:18.013239146 +0000 UTC m=+672.191098257" watchObservedRunningTime="2026-07-06 16:49:18.01426786 +0000 UTC m=+672.192126972" Jul 06 16:49:18.038702 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:18.038656 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5548f7d4db-lcsdz" podStartSLOduration=1.446956571 podStartE2EDuration="7.038640378s" totalImagesPullingTime="5.591683807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:49:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:49:18.03841365 +0000 UTC m=+672.216272762" watchObservedRunningTime="2026-07-06 16:49:18.038640378 +0000 UTC m=+672.216499495" Jul 06 16:49:19.485107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.485073 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6"] Jul 06 16:49:19.488571 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.488553 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.491185 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.491166 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-d4wqd\"" Jul 06 16:49:19.499936 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.499917 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 16:49:19.575272 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.575238 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6"] Jul 06 16:49:19.629318 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.629279 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.629500 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.629334 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.629500 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.629360 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.629500 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.629433 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.629614 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.629502 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.629614 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.629519 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.629614 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.629596 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.629705 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.629652 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.629744 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.629704 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqsss\" (UniqueName: \"kubernetes.io/projected/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-kube-api-access-qqsss\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.737540 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.731359 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.737540 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.731417 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.737540 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.731463 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.737540 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.731536 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.737540 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.731583 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqsss\" (UniqueName: \"kubernetes.io/projected/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-kube-api-access-qqsss\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.737540 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.731635 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.737540 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.731661 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.737540 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.731699 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.737540 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.731747 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.737540 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.732558 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.737540 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.732875 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.737540 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.733152 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.738223 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.737741 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.738607 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.738413 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.738909 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.738889 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.740551 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.740526 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.743408 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.743379 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.746272 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.746237 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqsss\" (UniqueName: \"kubernetes.io/projected/3a834f5d-a321-49e6-afe4-5d054c1ee3fc-kube-api-access-qqsss\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678gfmz6\" (UID: \"3a834f5d-a321-49e6-afe4-5d054c1ee3fc\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.798859 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.798832 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:19.951191 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.951165 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6"] Jul 06 16:49:19.953243 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:49:19.953218 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a834f5d_a321_49e6_afe4_5d054c1ee3fc.slice/crio-8ffcfe54649a2fb6c548acfef7ab28672c28311266d634460339f68d029d4891 WatchSource:0}: Error finding container 8ffcfe54649a2fb6c548acfef7ab28672c28311266d634460339f68d029d4891: Status 404 returned error can't find the container with id 8ffcfe54649a2fb6c548acfef7ab28672c28311266d634460339f68d029d4891 Jul 06 16:49:19.984565 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:19.984537 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" event={"ID":"3a834f5d-a321-49e6-afe4-5d054c1ee3fc","Type":"ContainerStarted","Data":"8ffcfe54649a2fb6c548acfef7ab28672c28311266d634460339f68d029d4891"} Jul 06 16:49:21.237348 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.237310 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm"] Jul 06 16:49:21.241249 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.241227 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm" Jul 06 16:49:21.245820 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.245796 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 16:49:21.247404 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.247382 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 16:49:21.248996 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.248975 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-g4s4q\"" Jul 06 16:49:21.252748 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.252730 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 16:49:21.301630 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.301598 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm"] Jul 06 16:49:21.346852 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.346814 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6a6976fd-d3e3-47c7-b419-55facb3761a5-manager-config\") pod \"lws-controller-manager-5c47b9fd86-qfqfm\" (UID: \"6a6976fd-d3e3-47c7-b419-55facb3761a5\") " pod="openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm" Jul 06 16:49:21.347022 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.346922 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a6976fd-d3e3-47c7-b419-55facb3761a5-metrics-cert\") pod \"lws-controller-manager-5c47b9fd86-qfqfm\" (UID: \"6a6976fd-d3e3-47c7-b419-55facb3761a5\") " pod="openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm" Jul 06 16:49:21.347022 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.346957 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6a6976fd-d3e3-47c7-b419-55facb3761a5-cert\") pod \"lws-controller-manager-5c47b9fd86-qfqfm\" (UID: \"6a6976fd-d3e3-47c7-b419-55facb3761a5\") " pod="openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm" Jul 06 16:49:21.347113 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.347034 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sxqv\" (UniqueName: \"kubernetes.io/projected/6a6976fd-d3e3-47c7-b419-55facb3761a5-kube-api-access-8sxqv\") pod \"lws-controller-manager-5c47b9fd86-qfqfm\" (UID: \"6a6976fd-d3e3-47c7-b419-55facb3761a5\") " pod="openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm" Jul 06 16:49:21.447899 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.447863 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6a6976fd-d3e3-47c7-b419-55facb3761a5-manager-config\") pod \"lws-controller-manager-5c47b9fd86-qfqfm\" (UID: \"6a6976fd-d3e3-47c7-b419-55facb3761a5\") " pod="openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm" Jul 06 16:49:21.448078 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.447917 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a6976fd-d3e3-47c7-b419-55facb3761a5-metrics-cert\") pod \"lws-controller-manager-5c47b9fd86-qfqfm\" (UID: \"6a6976fd-d3e3-47c7-b419-55facb3761a5\") " pod="openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm" Jul 06 16:49:21.448078 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.447938 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6a6976fd-d3e3-47c7-b419-55facb3761a5-cert\") pod \"lws-controller-manager-5c47b9fd86-qfqfm\" (UID: \"6a6976fd-d3e3-47c7-b419-55facb3761a5\") " pod="openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm" Jul 06 16:49:21.448078 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.447977 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sxqv\" (UniqueName: \"kubernetes.io/projected/6a6976fd-d3e3-47c7-b419-55facb3761a5-kube-api-access-8sxqv\") pod \"lws-controller-manager-5c47b9fd86-qfqfm\" (UID: \"6a6976fd-d3e3-47c7-b419-55facb3761a5\") " pod="openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm" Jul 06 16:49:21.448593 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.448569 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/6a6976fd-d3e3-47c7-b419-55facb3761a5-manager-config\") pod \"lws-controller-manager-5c47b9fd86-qfqfm\" (UID: \"6a6976fd-d3e3-47c7-b419-55facb3761a5\") " pod="openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm" Jul 06 16:49:21.450383 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.450362 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/6a6976fd-d3e3-47c7-b419-55facb3761a5-metrics-cert\") pod \"lws-controller-manager-5c47b9fd86-qfqfm\" (UID: \"6a6976fd-d3e3-47c7-b419-55facb3761a5\") " pod="openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm" Jul 06 16:49:21.450619 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.450594 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6a6976fd-d3e3-47c7-b419-55facb3761a5-cert\") pod \"lws-controller-manager-5c47b9fd86-qfqfm\" (UID: \"6a6976fd-d3e3-47c7-b419-55facb3761a5\") " pod="openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm" Jul 06 16:49:21.462334 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.462313 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sxqv\" (UniqueName: \"kubernetes.io/projected/6a6976fd-d3e3-47c7-b419-55facb3761a5-kube-api-access-8sxqv\") pod \"lws-controller-manager-5c47b9fd86-qfqfm\" (UID: \"6a6976fd-d3e3-47c7-b419-55facb3761a5\") " pod="openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm" Jul 06 16:49:21.553089 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.553013 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm" Jul 06 16:49:21.704532 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.701785 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm"] Jul 06 16:49:21.997596 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:21.997552 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm" event={"ID":"6a6976fd-d3e3-47c7-b419-55facb3761a5","Type":"ContainerStarted","Data":"ab59f5fa719f37b435c55c21f34b003228a32df0bf7b6ec108133e2a12aace56"} Jul 06 16:49:22.981568 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:22.981524 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 16:49:22.981945 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:22.981641 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 16:49:22.981945 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:22.981681 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 06 16:49:24.012677 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:24.012641 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" event={"ID":"3a834f5d-a321-49e6-afe4-5d054c1ee3fc","Type":"ContainerStarted","Data":"34a3b65cacaae03e1406d674b3cb2367d93e5d2a541eed8b176d96f442aab5f8"} Jul 06 16:49:24.064075 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:24.064025 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" podStartSLOduration=2.038110896 podStartE2EDuration="5.064007645s" totalImagesPullingTime="3.025896749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:49:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:49:24.061963895 +0000 UTC m=+678.239823009" watchObservedRunningTime="2026-07-06 16:49:24.064007645 +0000 UTC m=+678.241866756" Jul 06 16:49:24.799179 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:24.799148 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:24.805089 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:24.805061 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:25.018580 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:25.018544 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm" event={"ID":"6a6976fd-d3e3-47c7-b419-55facb3761a5","Type":"ContainerStarted","Data":"ede5cd9173fc1d253b6cfba7910b0f65c1828c97e2cfa9997e81eec5cda99c5a"} Jul 06 16:49:25.019037 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:25.018815 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:25.019746 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:25.019725 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678gfmz6" Jul 06 16:49:25.045653 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:25.045616 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm" podStartSLOduration=1.6453622129999999 podStartE2EDuration="4.045602563s" totalImagesPullingTime="2.40024035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:49:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:49:25.044854453 +0000 UTC m=+679.222713566" watchObservedRunningTime="2026-07-06 16:49:25.045602563 +0000 UTC m=+679.223461676" Jul 06 16:49:26.029821 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:26.026213 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm" Jul 06 16:49:28.981098 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:28.981066 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-576h9" Jul 06 16:49:28.983825 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:28.983805 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-blccs" Jul 06 16:49:37.031866 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:49:37.031837 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-5c47b9fd86-qfqfm" Jul 06 16:50:28.048954 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.048916 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx"] Jul 06 16:50:28.052651 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.052634 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx" Jul 06 16:50:28.055192 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.055165 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 16:50:28.055324 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.055199 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-sfj6x\"" Jul 06 16:50:28.055624 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.055607 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 16:50:28.063618 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.063596 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx"] Jul 06 16:50:28.224920 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.224884 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9gj2\" (UniqueName: \"kubernetes.io/projected/dcfa17b3-63be-4f41-925b-adf8dd903fa7-kube-api-access-k9gj2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx\" (UID: \"dcfa17b3-63be-4f41-925b-adf8dd903fa7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx" Jul 06 16:50:28.225084 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.224945 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcfa17b3-63be-4f41-925b-adf8dd903fa7-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx\" (UID: \"dcfa17b3-63be-4f41-925b-adf8dd903fa7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx" Jul 06 16:50:28.225084 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.224998 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcfa17b3-63be-4f41-925b-adf8dd903fa7-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx\" (UID: \"dcfa17b3-63be-4f41-925b-adf8dd903fa7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx" Jul 06 16:50:28.326409 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.326316 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcfa17b3-63be-4f41-925b-adf8dd903fa7-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx\" (UID: \"dcfa17b3-63be-4f41-925b-adf8dd903fa7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx" Jul 06 16:50:28.326409 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.326351 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcfa17b3-63be-4f41-925b-adf8dd903fa7-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx\" (UID: \"dcfa17b3-63be-4f41-925b-adf8dd903fa7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx" Jul 06 16:50:28.326648 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.326436 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9gj2\" (UniqueName: \"kubernetes.io/projected/dcfa17b3-63be-4f41-925b-adf8dd903fa7-kube-api-access-k9gj2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx\" (UID: \"dcfa17b3-63be-4f41-925b-adf8dd903fa7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx" Jul 06 16:50:28.326801 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.326765 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcfa17b3-63be-4f41-925b-adf8dd903fa7-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx\" (UID: \"dcfa17b3-63be-4f41-925b-adf8dd903fa7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx" Jul 06 16:50:28.326865 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.326775 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcfa17b3-63be-4f41-925b-adf8dd903fa7-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx\" (UID: \"dcfa17b3-63be-4f41-925b-adf8dd903fa7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx" Jul 06 16:50:28.335967 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.335944 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9gj2\" (UniqueName: \"kubernetes.io/projected/dcfa17b3-63be-4f41-925b-adf8dd903fa7-kube-api-access-k9gj2\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx\" (UID: \"dcfa17b3-63be-4f41-925b-adf8dd903fa7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx" Jul 06 16:50:28.361756 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.361727 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx" Jul 06 16:50:28.493728 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.493705 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx"] Jul 06 16:50:28.495778 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:50:28.495747 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcfa17b3_63be_4f41_925b_adf8dd903fa7.slice/crio-eb0fe69d5c5e7692375deccb7211483ccf2556b6ae0ab9dcbdfa5a020e721524 WatchSource:0}: Error finding container eb0fe69d5c5e7692375deccb7211483ccf2556b6ae0ab9dcbdfa5a020e721524: Status 404 returned error can't find the container with id eb0fe69d5c5e7692375deccb7211483ccf2556b6ae0ab9dcbdfa5a020e721524 Jul 06 16:50:28.635453 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.635422 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc"] Jul 06 16:50:28.638862 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.638844 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc" Jul 06 16:50:28.648433 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.648322 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc"] Jul 06 16:50:28.830449 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.830412 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abcfeaa2-8dc0-45bc-a4b3-d96654b78504-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc\" (UID: \"abcfeaa2-8dc0-45bc-a4b3-d96654b78504\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc" Jul 06 16:50:28.830677 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.830506 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4lfr\" (UniqueName: \"kubernetes.io/projected/abcfeaa2-8dc0-45bc-a4b3-d96654b78504-kube-api-access-q4lfr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc\" (UID: \"abcfeaa2-8dc0-45bc-a4b3-d96654b78504\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc" Jul 06 16:50:28.830677 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.830583 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abcfeaa2-8dc0-45bc-a4b3-d96654b78504-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc\" (UID: \"abcfeaa2-8dc0-45bc-a4b3-d96654b78504\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc" Jul 06 16:50:28.931518 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.931415 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abcfeaa2-8dc0-45bc-a4b3-d96654b78504-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc\" (UID: \"abcfeaa2-8dc0-45bc-a4b3-d96654b78504\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc" Jul 06 16:50:28.931518 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.931482 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4lfr\" (UniqueName: \"kubernetes.io/projected/abcfeaa2-8dc0-45bc-a4b3-d96654b78504-kube-api-access-q4lfr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc\" (UID: \"abcfeaa2-8dc0-45bc-a4b3-d96654b78504\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc" Jul 06 16:50:28.931743 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.931527 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abcfeaa2-8dc0-45bc-a4b3-d96654b78504-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc\" (UID: \"abcfeaa2-8dc0-45bc-a4b3-d96654b78504\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc" Jul 06 16:50:28.931907 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.931884 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abcfeaa2-8dc0-45bc-a4b3-d96654b78504-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc\" (UID: \"abcfeaa2-8dc0-45bc-a4b3-d96654b78504\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc" Jul 06 16:50:28.931907 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.931900 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abcfeaa2-8dc0-45bc-a4b3-d96654b78504-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc\" (UID: \"abcfeaa2-8dc0-45bc-a4b3-d96654b78504\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc" Jul 06 16:50:28.940757 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.940735 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4lfr\" (UniqueName: \"kubernetes.io/projected/abcfeaa2-8dc0-45bc-a4b3-d96654b78504-kube-api-access-q4lfr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc\" (UID: \"abcfeaa2-8dc0-45bc-a4b3-d96654b78504\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc" Jul 06 16:50:28.968967 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:28.968940 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc" Jul 06 16:50:29.055679 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.055649 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp"] Jul 06 16:50:29.059613 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.059593 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp" Jul 06 16:50:29.069935 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.069857 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp"] Jul 06 16:50:29.114338 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.114311 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc"] Jul 06 16:50:29.115820 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:50:29.115784 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabcfeaa2_8dc0_45bc_a4b3_d96654b78504.slice/crio-39ef6bc45bafb626b8645c8b7182101a93443d683406d24214c9770fe388111c WatchSource:0}: Error finding container 39ef6bc45bafb626b8645c8b7182101a93443d683406d24214c9770fe388111c: Status 404 returned error can't find the container with id 39ef6bc45bafb626b8645c8b7182101a93443d683406d24214c9770fe388111c Jul 06 16:50:29.233358 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.233328 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/743eeb1b-83ee-435d-886e-bed15f92ea77-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp\" (UID: \"743eeb1b-83ee-435d-886e-bed15f92ea77\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp" Jul 06 16:50:29.233521 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.233373 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnh66\" (UniqueName: \"kubernetes.io/projected/743eeb1b-83ee-435d-886e-bed15f92ea77-kube-api-access-jnh66\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp\" (UID: \"743eeb1b-83ee-435d-886e-bed15f92ea77\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp" Jul 06 16:50:29.233521 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.233459 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/743eeb1b-83ee-435d-886e-bed15f92ea77-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp\" (UID: \"743eeb1b-83ee-435d-886e-bed15f92ea77\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp" Jul 06 16:50:29.326713 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.326681 2580 generic.go:356] "Generic (PLEG): container finished" podID="abcfeaa2-8dc0-45bc-a4b3-d96654b78504" containerID="b1e694a44465253fb0bc6fbf9cbd4ebd59d002bc68d65be2989a1b74ec4b062d" exitCode=0 Jul 06 16:50:29.326877 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.326772 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc" event={"ID":"abcfeaa2-8dc0-45bc-a4b3-d96654b78504","Type":"ContainerDied","Data":"b1e694a44465253fb0bc6fbf9cbd4ebd59d002bc68d65be2989a1b74ec4b062d"} Jul 06 16:50:29.326877 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.326829 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc" event={"ID":"abcfeaa2-8dc0-45bc-a4b3-d96654b78504","Type":"ContainerStarted","Data":"39ef6bc45bafb626b8645c8b7182101a93443d683406d24214c9770fe388111c"} Jul 06 16:50:29.328404 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.328379 2580 generic.go:356] "Generic (PLEG): container finished" podID="dcfa17b3-63be-4f41-925b-adf8dd903fa7" containerID="f826df10780d801c4558876fd289674d63db71d78e47ceaa3a2377e753e7b746" exitCode=0 Jul 06 16:50:29.328526 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.328418 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx" event={"ID":"dcfa17b3-63be-4f41-925b-adf8dd903fa7","Type":"ContainerDied","Data":"f826df10780d801c4558876fd289674d63db71d78e47ceaa3a2377e753e7b746"} Jul 06 16:50:29.328526 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.328491 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx" event={"ID":"dcfa17b3-63be-4f41-925b-adf8dd903fa7","Type":"ContainerStarted","Data":"eb0fe69d5c5e7692375deccb7211483ccf2556b6ae0ab9dcbdfa5a020e721524"} Jul 06 16:50:29.334856 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.334836 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/743eeb1b-83ee-435d-886e-bed15f92ea77-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp\" (UID: \"743eeb1b-83ee-435d-886e-bed15f92ea77\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp" Jul 06 16:50:29.334952 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.334890 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/743eeb1b-83ee-435d-886e-bed15f92ea77-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp\" (UID: \"743eeb1b-83ee-435d-886e-bed15f92ea77\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp" Jul 06 16:50:29.334952 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.334914 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnh66\" (UniqueName: \"kubernetes.io/projected/743eeb1b-83ee-435d-886e-bed15f92ea77-kube-api-access-jnh66\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp\" (UID: \"743eeb1b-83ee-435d-886e-bed15f92ea77\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp" Jul 06 16:50:29.335252 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.335233 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/743eeb1b-83ee-435d-886e-bed15f92ea77-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp\" (UID: \"743eeb1b-83ee-435d-886e-bed15f92ea77\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp" Jul 06 16:50:29.335296 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.335246 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/743eeb1b-83ee-435d-886e-bed15f92ea77-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp\" (UID: \"743eeb1b-83ee-435d-886e-bed15f92ea77\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp" Jul 06 16:50:29.341954 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.341935 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnh66\" (UniqueName: \"kubernetes.io/projected/743eeb1b-83ee-435d-886e-bed15f92ea77-kube-api-access-jnh66\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp\" (UID: \"743eeb1b-83ee-435d-886e-bed15f92ea77\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp" Jul 06 16:50:29.369658 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.369619 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp" Jul 06 16:50:29.466917 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.466849 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw"] Jul 06 16:50:29.470719 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.470697 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw" Jul 06 16:50:29.484774 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.484746 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw"] Jul 06 16:50:29.512650 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.512628 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp"] Jul 06 16:50:29.515776 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:50:29.515754 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod743eeb1b_83ee_435d_886e_bed15f92ea77.slice/crio-b1c18a4c91a2c0390c5dae413e2aff27bf0b81862bdd097f8d79bd1f020d0501 WatchSource:0}: Error finding container b1c18a4c91a2c0390c5dae413e2aff27bf0b81862bdd097f8d79bd1f020d0501: Status 404 returned error can't find the container with id b1c18a4c91a2c0390c5dae413e2aff27bf0b81862bdd097f8d79bd1f020d0501 Jul 06 16:50:29.636884 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.636859 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb983e83-732f-4b99-bfa8-8b774db1f6d5-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw\" (UID: \"bb983e83-732f-4b99-bfa8-8b774db1f6d5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw" Jul 06 16:50:29.636996 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.636904 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb983e83-732f-4b99-bfa8-8b774db1f6d5-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw\" (UID: \"bb983e83-732f-4b99-bfa8-8b774db1f6d5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw" Jul 06 16:50:29.636996 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.636925 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nztq9\" (UniqueName: \"kubernetes.io/projected/bb983e83-732f-4b99-bfa8-8b774db1f6d5-kube-api-access-nztq9\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw\" (UID: \"bb983e83-732f-4b99-bfa8-8b774db1f6d5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw" Jul 06 16:50:29.738184 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.738117 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb983e83-732f-4b99-bfa8-8b774db1f6d5-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw\" (UID: \"bb983e83-732f-4b99-bfa8-8b774db1f6d5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw" Jul 06 16:50:29.738184 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.738164 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb983e83-732f-4b99-bfa8-8b774db1f6d5-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw\" (UID: \"bb983e83-732f-4b99-bfa8-8b774db1f6d5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw" Jul 06 16:50:29.738350 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.738189 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nztq9\" (UniqueName: \"kubernetes.io/projected/bb983e83-732f-4b99-bfa8-8b774db1f6d5-kube-api-access-nztq9\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw\" (UID: \"bb983e83-732f-4b99-bfa8-8b774db1f6d5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw" Jul 06 16:50:29.738608 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.738587 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb983e83-732f-4b99-bfa8-8b774db1f6d5-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw\" (UID: \"bb983e83-732f-4b99-bfa8-8b774db1f6d5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw" Jul 06 16:50:29.738694 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.738617 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb983e83-732f-4b99-bfa8-8b774db1f6d5-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw\" (UID: \"bb983e83-732f-4b99-bfa8-8b774db1f6d5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw" Jul 06 16:50:29.747820 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.747797 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nztq9\" (UniqueName: \"kubernetes.io/projected/bb983e83-732f-4b99-bfa8-8b774db1f6d5-kube-api-access-nztq9\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw\" (UID: \"bb983e83-732f-4b99-bfa8-8b774db1f6d5\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw" Jul 06 16:50:29.782119 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.782099 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw" Jul 06 16:50:29.915246 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:29.915220 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw"] Jul 06 16:50:29.916608 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:50:29.916584 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb983e83_732f_4b99_bfa8_8b774db1f6d5.slice/crio-5aa431488ace5b563297fdafab0e6ff525247353718e7ad7b7c5e0455276ce89 WatchSource:0}: Error finding container 5aa431488ace5b563297fdafab0e6ff525247353718e7ad7b7c5e0455276ce89: Status 404 returned error can't find the container with id 5aa431488ace5b563297fdafab0e6ff525247353718e7ad7b7c5e0455276ce89 Jul 06 16:50:30.335847 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:30.335818 2580 generic.go:356] "Generic (PLEG): container finished" podID="dcfa17b3-63be-4f41-925b-adf8dd903fa7" containerID="3d19ca2d77bdf0c4e394712070efe5fdcadc4d3f7065aa5f005569ae37be401b" exitCode=0 Jul 06 16:50:30.336297 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:30.335895 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx" event={"ID":"dcfa17b3-63be-4f41-925b-adf8dd903fa7","Type":"ContainerDied","Data":"3d19ca2d77bdf0c4e394712070efe5fdcadc4d3f7065aa5f005569ae37be401b"} Jul 06 16:50:30.337638 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:30.337609 2580 generic.go:356] "Generic (PLEG): container finished" podID="bb983e83-732f-4b99-bfa8-8b774db1f6d5" containerID="fd2148783c6e0f5e7f86e25866bc90cd5b44750946ff79e7c62944222d2e1e5b" exitCode=0 Jul 06 16:50:30.337783 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:30.337695 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw" event={"ID":"bb983e83-732f-4b99-bfa8-8b774db1f6d5","Type":"ContainerDied","Data":"fd2148783c6e0f5e7f86e25866bc90cd5b44750946ff79e7c62944222d2e1e5b"} Jul 06 16:50:30.337783 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:30.337738 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw" event={"ID":"bb983e83-732f-4b99-bfa8-8b774db1f6d5","Type":"ContainerStarted","Data":"5aa431488ace5b563297fdafab0e6ff525247353718e7ad7b7c5e0455276ce89"} Jul 06 16:50:30.339449 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:30.339428 2580 generic.go:356] "Generic (PLEG): container finished" podID="743eeb1b-83ee-435d-886e-bed15f92ea77" containerID="e9938324a3c2d6e088b40dc2617d7b4c454e9598790667ee972100c880801c5b" exitCode=0 Jul 06 16:50:30.339588 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:30.339500 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp" event={"ID":"743eeb1b-83ee-435d-886e-bed15f92ea77","Type":"ContainerDied","Data":"e9938324a3c2d6e088b40dc2617d7b4c454e9598790667ee972100c880801c5b"} Jul 06 16:50:30.339588 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:30.339535 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp" event={"ID":"743eeb1b-83ee-435d-886e-bed15f92ea77","Type":"ContainerStarted","Data":"b1c18a4c91a2c0390c5dae413e2aff27bf0b81862bdd097f8d79bd1f020d0501"} Jul 06 16:50:31.346692 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:31.346653 2580 generic.go:356] "Generic (PLEG): container finished" podID="dcfa17b3-63be-4f41-925b-adf8dd903fa7" containerID="76355e3060d59b8e7436ccf53924431f7ad22c36af5eeb455681c8cb0d0fea6c" exitCode=0 Jul 06 16:50:31.347108 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:31.346738 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx" event={"ID":"dcfa17b3-63be-4f41-925b-adf8dd903fa7","Type":"ContainerDied","Data":"76355e3060d59b8e7436ccf53924431f7ad22c36af5eeb455681c8cb0d0fea6c"} Jul 06 16:50:31.348383 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:31.348356 2580 generic.go:356] "Generic (PLEG): container finished" podID="bb983e83-732f-4b99-bfa8-8b774db1f6d5" containerID="44094b93ac4374ba31a8b687b68b29e1b48db7e4b361ea9ef08df4cd8909a0c2" exitCode=0 Jul 06 16:50:31.348546 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:31.348445 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw" event={"ID":"bb983e83-732f-4b99-bfa8-8b774db1f6d5","Type":"ContainerDied","Data":"44094b93ac4374ba31a8b687b68b29e1b48db7e4b361ea9ef08df4cd8909a0c2"} Jul 06 16:50:31.350300 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:31.350275 2580 generic.go:356] "Generic (PLEG): container finished" podID="abcfeaa2-8dc0-45bc-a4b3-d96654b78504" containerID="dcd792f0bc909258a68b585ba815d61d411d262d776c8562f9e1c9e89e92f019" exitCode=0 Jul 06 16:50:31.350397 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:31.350321 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc" event={"ID":"abcfeaa2-8dc0-45bc-a4b3-d96654b78504","Type":"ContainerDied","Data":"dcd792f0bc909258a68b585ba815d61d411d262d776c8562f9e1c9e89e92f019"} Jul 06 16:50:32.357249 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:32.357214 2580 generic.go:356] "Generic (PLEG): container finished" podID="abcfeaa2-8dc0-45bc-a4b3-d96654b78504" containerID="2bc662e164661d7d27d755407825b372f2dfb16b81880008fd108c4512acf438" exitCode=0 Jul 06 16:50:32.357706 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:32.357269 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc" event={"ID":"abcfeaa2-8dc0-45bc-a4b3-d96654b78504","Type":"ContainerDied","Data":"2bc662e164661d7d27d755407825b372f2dfb16b81880008fd108c4512acf438"} Jul 06 16:50:32.359019 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:32.358993 2580 generic.go:356] "Generic (PLEG): container finished" podID="743eeb1b-83ee-435d-886e-bed15f92ea77" containerID="746ac70864543a51240f755ce51e17d77fcb01b86e2f88f223a3dfcc8f86ba9e" exitCode=0 Jul 06 16:50:32.359130 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:32.359070 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp" event={"ID":"743eeb1b-83ee-435d-886e-bed15f92ea77","Type":"ContainerDied","Data":"746ac70864543a51240f755ce51e17d77fcb01b86e2f88f223a3dfcc8f86ba9e"} Jul 06 16:50:32.361570 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:32.361549 2580 generic.go:356] "Generic (PLEG): container finished" podID="bb983e83-732f-4b99-bfa8-8b774db1f6d5" containerID="4ec5a005470f13468ca44649594ea96168cf87cf0a93795c12003c71c36ecff9" exitCode=0 Jul 06 16:50:32.361686 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:32.361622 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw" event={"ID":"bb983e83-732f-4b99-bfa8-8b774db1f6d5","Type":"ContainerDied","Data":"4ec5a005470f13468ca44649594ea96168cf87cf0a93795c12003c71c36ecff9"} Jul 06 16:50:32.500371 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:32.500346 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx" Jul 06 16:50:32.561605 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:32.561576 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcfa17b3-63be-4f41-925b-adf8dd903fa7-bundle\") pod \"dcfa17b3-63be-4f41-925b-adf8dd903fa7\" (UID: \"dcfa17b3-63be-4f41-925b-adf8dd903fa7\") " Jul 06 16:50:32.561724 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:32.561610 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcfa17b3-63be-4f41-925b-adf8dd903fa7-util\") pod \"dcfa17b3-63be-4f41-925b-adf8dd903fa7\" (UID: \"dcfa17b3-63be-4f41-925b-adf8dd903fa7\") " Jul 06 16:50:32.561724 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:32.561644 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k9gj2\" (UniqueName: \"kubernetes.io/projected/dcfa17b3-63be-4f41-925b-adf8dd903fa7-kube-api-access-k9gj2\") pod \"dcfa17b3-63be-4f41-925b-adf8dd903fa7\" (UID: \"dcfa17b3-63be-4f41-925b-adf8dd903fa7\") " Jul 06 16:50:32.562044 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:32.562018 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcfa17b3-63be-4f41-925b-adf8dd903fa7-bundle" (OuterVolumeSpecName: "bundle") pod "dcfa17b3-63be-4f41-925b-adf8dd903fa7" (UID: "dcfa17b3-63be-4f41-925b-adf8dd903fa7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:50:32.563704 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:32.563681 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcfa17b3-63be-4f41-925b-adf8dd903fa7-kube-api-access-k9gj2" (OuterVolumeSpecName: "kube-api-access-k9gj2") pod "dcfa17b3-63be-4f41-925b-adf8dd903fa7" (UID: "dcfa17b3-63be-4f41-925b-adf8dd903fa7"). InnerVolumeSpecName "kube-api-access-k9gj2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:50:32.567549 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:32.567528 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dcfa17b3-63be-4f41-925b-adf8dd903fa7-util" (OuterVolumeSpecName: "util") pod "dcfa17b3-63be-4f41-925b-adf8dd903fa7" (UID: "dcfa17b3-63be-4f41-925b-adf8dd903fa7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:50:32.662732 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:32.662651 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dcfa17b3-63be-4f41-925b-adf8dd903fa7-bundle\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:50:32.662732 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:32.662682 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dcfa17b3-63be-4f41-925b-adf8dd903fa7-util\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:50:32.662732 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:32.662695 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9gj2\" (UniqueName: \"kubernetes.io/projected/dcfa17b3-63be-4f41-925b-adf8dd903fa7-kube-api-access-k9gj2\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:50:33.368839 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.368803 2580 generic.go:356] "Generic (PLEG): container finished" podID="743eeb1b-83ee-435d-886e-bed15f92ea77" containerID="21e49964d2b3f684e15fadf62ffdf5ad5d3c6c8e17622813dd466bf83644009d" exitCode=0 Jul 06 16:50:33.369266 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.368893 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp" event={"ID":"743eeb1b-83ee-435d-886e-bed15f92ea77","Type":"ContainerDied","Data":"21e49964d2b3f684e15fadf62ffdf5ad5d3c6c8e17622813dd466bf83644009d"} Jul 06 16:50:33.370770 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.370745 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx" event={"ID":"dcfa17b3-63be-4f41-925b-adf8dd903fa7","Type":"ContainerDied","Data":"eb0fe69d5c5e7692375deccb7211483ccf2556b6ae0ab9dcbdfa5a020e721524"} Jul 06 16:50:33.370864 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.370767 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx" Jul 06 16:50:33.370864 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.370785 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb0fe69d5c5e7692375deccb7211483ccf2556b6ae0ab9dcbdfa5a020e721524" Jul 06 16:50:33.535835 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.535797 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc" Jul 06 16:50:33.538374 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.538354 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw" Jul 06 16:50:33.571137 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.571112 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb983e83-732f-4b99-bfa8-8b774db1f6d5-util\") pod \"bb983e83-732f-4b99-bfa8-8b774db1f6d5\" (UID: \"bb983e83-732f-4b99-bfa8-8b774db1f6d5\") " Jul 06 16:50:33.571261 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.571156 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nztq9\" (UniqueName: \"kubernetes.io/projected/bb983e83-732f-4b99-bfa8-8b774db1f6d5-kube-api-access-nztq9\") pod \"bb983e83-732f-4b99-bfa8-8b774db1f6d5\" (UID: \"bb983e83-732f-4b99-bfa8-8b774db1f6d5\") " Jul 06 16:50:33.571261 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.571177 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abcfeaa2-8dc0-45bc-a4b3-d96654b78504-bundle\") pod \"abcfeaa2-8dc0-45bc-a4b3-d96654b78504\" (UID: \"abcfeaa2-8dc0-45bc-a4b3-d96654b78504\") " Jul 06 16:50:33.571261 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.571199 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb983e83-732f-4b99-bfa8-8b774db1f6d5-bundle\") pod \"bb983e83-732f-4b99-bfa8-8b774db1f6d5\" (UID: \"bb983e83-732f-4b99-bfa8-8b774db1f6d5\") " Jul 06 16:50:33.571261 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.571253 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4lfr\" (UniqueName: \"kubernetes.io/projected/abcfeaa2-8dc0-45bc-a4b3-d96654b78504-kube-api-access-q4lfr\") pod \"abcfeaa2-8dc0-45bc-a4b3-d96654b78504\" (UID: \"abcfeaa2-8dc0-45bc-a4b3-d96654b78504\") " Jul 06 16:50:33.571483 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.571292 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abcfeaa2-8dc0-45bc-a4b3-d96654b78504-util\") pod \"abcfeaa2-8dc0-45bc-a4b3-d96654b78504\" (UID: \"abcfeaa2-8dc0-45bc-a4b3-d96654b78504\") " Jul 06 16:50:33.571718 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.571689 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abcfeaa2-8dc0-45bc-a4b3-d96654b78504-bundle" (OuterVolumeSpecName: "bundle") pod "abcfeaa2-8dc0-45bc-a4b3-d96654b78504" (UID: "abcfeaa2-8dc0-45bc-a4b3-d96654b78504"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:50:33.573827 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.573725 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb983e83-732f-4b99-bfa8-8b774db1f6d5-bundle" (OuterVolumeSpecName: "bundle") pod "bb983e83-732f-4b99-bfa8-8b774db1f6d5" (UID: "bb983e83-732f-4b99-bfa8-8b774db1f6d5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:50:33.574333 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.574295 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb983e83-732f-4b99-bfa8-8b774db1f6d5-kube-api-access-nztq9" (OuterVolumeSpecName: "kube-api-access-nztq9") pod "bb983e83-732f-4b99-bfa8-8b774db1f6d5" (UID: "bb983e83-732f-4b99-bfa8-8b774db1f6d5"). InnerVolumeSpecName "kube-api-access-nztq9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:50:33.574333 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.574312 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abcfeaa2-8dc0-45bc-a4b3-d96654b78504-kube-api-access-q4lfr" (OuterVolumeSpecName: "kube-api-access-q4lfr") pod "abcfeaa2-8dc0-45bc-a4b3-d96654b78504" (UID: "abcfeaa2-8dc0-45bc-a4b3-d96654b78504"). InnerVolumeSpecName "kube-api-access-q4lfr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:50:33.579381 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.579255 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb983e83-732f-4b99-bfa8-8b774db1f6d5-util" (OuterVolumeSpecName: "util") pod "bb983e83-732f-4b99-bfa8-8b774db1f6d5" (UID: "bb983e83-732f-4b99-bfa8-8b774db1f6d5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:50:33.581574 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.581546 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abcfeaa2-8dc0-45bc-a4b3-d96654b78504-util" (OuterVolumeSpecName: "util") pod "abcfeaa2-8dc0-45bc-a4b3-d96654b78504" (UID: "abcfeaa2-8dc0-45bc-a4b3-d96654b78504"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:50:33.672947 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.672862 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bb983e83-732f-4b99-bfa8-8b774db1f6d5-util\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:50:33.672947 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.672897 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nztq9\" (UniqueName: \"kubernetes.io/projected/bb983e83-732f-4b99-bfa8-8b774db1f6d5-kube-api-access-nztq9\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:50:33.672947 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.672909 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abcfeaa2-8dc0-45bc-a4b3-d96654b78504-bundle\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:50:33.672947 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.672917 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bb983e83-732f-4b99-bfa8-8b774db1f6d5-bundle\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:50:33.672947 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.672925 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4lfr\" (UniqueName: \"kubernetes.io/projected/abcfeaa2-8dc0-45bc-a4b3-d96654b78504-kube-api-access-q4lfr\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:50:33.672947 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:33.672933 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abcfeaa2-8dc0-45bc-a4b3-d96654b78504-util\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:50:34.377348 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:34.377319 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc" Jul 06 16:50:34.377804 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:34.377347 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc" event={"ID":"abcfeaa2-8dc0-45bc-a4b3-d96654b78504","Type":"ContainerDied","Data":"39ef6bc45bafb626b8645c8b7182101a93443d683406d24214c9770fe388111c"} Jul 06 16:50:34.377804 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:34.377398 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="39ef6bc45bafb626b8645c8b7182101a93443d683406d24214c9770fe388111c" Jul 06 16:50:34.379566 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:34.379478 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw" event={"ID":"bb983e83-732f-4b99-bfa8-8b774db1f6d5","Type":"ContainerDied","Data":"5aa431488ace5b563297fdafab0e6ff525247353718e7ad7b7c5e0455276ce89"} Jul 06 16:50:34.379566 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:34.379524 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5aa431488ace5b563297fdafab0e6ff525247353718e7ad7b7c5e0455276ce89" Jul 06 16:50:34.379768 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:34.379693 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw" Jul 06 16:50:34.514264 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:34.514238 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp" Jul 06 16:50:34.577509 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:34.577407 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnh66\" (UniqueName: \"kubernetes.io/projected/743eeb1b-83ee-435d-886e-bed15f92ea77-kube-api-access-jnh66\") pod \"743eeb1b-83ee-435d-886e-bed15f92ea77\" (UID: \"743eeb1b-83ee-435d-886e-bed15f92ea77\") " Jul 06 16:50:34.577509 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:34.577454 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/743eeb1b-83ee-435d-886e-bed15f92ea77-util\") pod \"743eeb1b-83ee-435d-886e-bed15f92ea77\" (UID: \"743eeb1b-83ee-435d-886e-bed15f92ea77\") " Jul 06 16:50:34.577724 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:34.577549 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/743eeb1b-83ee-435d-886e-bed15f92ea77-bundle\") pod \"743eeb1b-83ee-435d-886e-bed15f92ea77\" (UID: \"743eeb1b-83ee-435d-886e-bed15f92ea77\") " Jul 06 16:50:34.578424 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:34.578394 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/743eeb1b-83ee-435d-886e-bed15f92ea77-bundle" (OuterVolumeSpecName: "bundle") pod "743eeb1b-83ee-435d-886e-bed15f92ea77" (UID: "743eeb1b-83ee-435d-886e-bed15f92ea77"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:50:34.579690 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:34.579662 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/743eeb1b-83ee-435d-886e-bed15f92ea77-kube-api-access-jnh66" (OuterVolumeSpecName: "kube-api-access-jnh66") pod "743eeb1b-83ee-435d-886e-bed15f92ea77" (UID: "743eeb1b-83ee-435d-886e-bed15f92ea77"). InnerVolumeSpecName "kube-api-access-jnh66". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:50:34.583320 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:34.583295 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/743eeb1b-83ee-435d-886e-bed15f92ea77-util" (OuterVolumeSpecName: "util") pod "743eeb1b-83ee-435d-886e-bed15f92ea77" (UID: "743eeb1b-83ee-435d-886e-bed15f92ea77"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 16:50:34.678701 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:34.678665 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnh66\" (UniqueName: \"kubernetes.io/projected/743eeb1b-83ee-435d-886e-bed15f92ea77-kube-api-access-jnh66\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:50:34.678701 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:34.678695 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/743eeb1b-83ee-435d-886e-bed15f92ea77-util\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:50:34.678701 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:34.678704 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/743eeb1b-83ee-435d-886e-bed15f92ea77-bundle\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:50:35.386707 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:35.386680 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp" Jul 06 16:50:35.387090 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:35.386712 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp" event={"ID":"743eeb1b-83ee-435d-886e-bed15f92ea77","Type":"ContainerDied","Data":"b1c18a4c91a2c0390c5dae413e2aff27bf0b81862bdd097f8d79bd1f020d0501"} Jul 06 16:50:35.387090 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:50:35.386755 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1c18a4c91a2c0390c5dae413e2aff27bf0b81862bdd097f8d79bd1f020d0501" Jul 06 16:51:57.331642 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.331610 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-hjssk"] Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.331947 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="743eeb1b-83ee-435d-886e-bed15f92ea77" containerName="pull" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.331958 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="743eeb1b-83ee-435d-886e-bed15f92ea77" containerName="pull" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.331967 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abcfeaa2-8dc0-45bc-a4b3-d96654b78504" containerName="pull" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.331972 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="abcfeaa2-8dc0-45bc-a4b3-d96654b78504" containerName="pull" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.331982 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcfa17b3-63be-4f41-925b-adf8dd903fa7" containerName="util" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.331987 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcfa17b3-63be-4f41-925b-adf8dd903fa7" containerName="util" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.331995 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abcfeaa2-8dc0-45bc-a4b3-d96654b78504" containerName="util" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.331999 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="abcfeaa2-8dc0-45bc-a4b3-d96654b78504" containerName="util" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.332009 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb983e83-732f-4b99-bfa8-8b774db1f6d5" containerName="util" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.332014 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb983e83-732f-4b99-bfa8-8b774db1f6d5" containerName="util" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.332025 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="743eeb1b-83ee-435d-886e-bed15f92ea77" containerName="extract" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.332030 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="743eeb1b-83ee-435d-886e-bed15f92ea77" containerName="extract" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.332038 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb983e83-732f-4b99-bfa8-8b774db1f6d5" containerName="pull" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.332042 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb983e83-732f-4b99-bfa8-8b774db1f6d5" containerName="pull" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.332048 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="743eeb1b-83ee-435d-886e-bed15f92ea77" containerName="util" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.332053 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="743eeb1b-83ee-435d-886e-bed15f92ea77" containerName="util" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.332058 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abcfeaa2-8dc0-45bc-a4b3-d96654b78504" containerName="extract" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.332063 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="abcfeaa2-8dc0-45bc-a4b3-d96654b78504" containerName="extract" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.332071 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcfa17b3-63be-4f41-925b-adf8dd903fa7" containerName="pull" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.332075 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcfa17b3-63be-4f41-925b-adf8dd903fa7" containerName="pull" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.332081 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcfa17b3-63be-4f41-925b-adf8dd903fa7" containerName="extract" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.332086 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcfa17b3-63be-4f41-925b-adf8dd903fa7" containerName="extract" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.332092 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb983e83-732f-4b99-bfa8-8b774db1f6d5" containerName="extract" Jul 06 16:51:57.332107 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.332097 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb983e83-732f-4b99-bfa8-8b774db1f6d5" containerName="extract" Jul 06 16:51:57.332872 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.332149 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb983e83-732f-4b99-bfa8-8b774db1f6d5" containerName="extract" Jul 06 16:51:57.332872 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.332156 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="743eeb1b-83ee-435d-886e-bed15f92ea77" containerName="extract" Jul 06 16:51:57.332872 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.332164 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcfa17b3-63be-4f41-925b-adf8dd903fa7" containerName="extract" Jul 06 16:51:57.332872 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.332173 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="abcfeaa2-8dc0-45bc-a4b3-d96654b78504" containerName="extract" Jul 06 16:51:57.335169 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.335153 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hjssk" Jul 06 16:51:57.337503 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.337479 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 16:51:57.337635 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.337510 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 16:51:57.337890 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.337874 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-69fsv\"" Jul 06 16:51:57.343185 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.343167 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-hjssk"] Jul 06 16:51:57.479543 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.479515 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw8jr\" (UniqueName: \"kubernetes.io/projected/09ec7040-20c0-44ef-9d2c-b717f2b6dd0b-kube-api-access-hw8jr\") pod \"authorino-58cd955675-hjssk\" (UID: \"09ec7040-20c0-44ef-9d2c-b717f2b6dd0b\") " pod="kuadrant-system/authorino-58cd955675-hjssk" Jul 06 16:51:57.580175 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.580148 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw8jr\" (UniqueName: \"kubernetes.io/projected/09ec7040-20c0-44ef-9d2c-b717f2b6dd0b-kube-api-access-hw8jr\") pod \"authorino-58cd955675-hjssk\" (UID: \"09ec7040-20c0-44ef-9d2c-b717f2b6dd0b\") " pod="kuadrant-system/authorino-58cd955675-hjssk" Jul 06 16:51:57.591132 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.591105 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw8jr\" (UniqueName: \"kubernetes.io/projected/09ec7040-20c0-44ef-9d2c-b717f2b6dd0b-kube-api-access-hw8jr\") pod \"authorino-58cd955675-hjssk\" (UID: \"09ec7040-20c0-44ef-9d2c-b717f2b6dd0b\") " pod="kuadrant-system/authorino-58cd955675-hjssk" Jul 06 16:51:57.646084 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.646058 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hjssk" Jul 06 16:51:57.702540 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.702512 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-47p5k"] Jul 06 16:51:57.706853 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.706832 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-47p5k" Jul 06 16:51:57.712115 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.712063 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-47p5k"] Jul 06 16:51:57.781945 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.781912 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb4qk\" (UniqueName: \"kubernetes.io/projected/89d3d741-e1fa-4702-a68a-0e7454c3e791-kube-api-access-tb4qk\") pod \"authorino-75d556bd77-47p5k\" (UID: \"89d3d741-e1fa-4702-a68a-0e7454c3e791\") " pod="kuadrant-system/authorino-75d556bd77-47p5k" Jul 06 16:51:57.789173 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.789152 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-hjssk"] Jul 06 16:51:57.791255 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:51:57.791226 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09ec7040_20c0_44ef_9d2c_b717f2b6dd0b.slice/crio-8923ab885717bc2c6f915cc0c206a647f3303f23b6fbf7a476d0ecd18ded3764 WatchSource:0}: Error finding container 8923ab885717bc2c6f915cc0c206a647f3303f23b6fbf7a476d0ecd18ded3764: Status 404 returned error can't find the container with id 8923ab885717bc2c6f915cc0c206a647f3303f23b6fbf7a476d0ecd18ded3764 Jul 06 16:51:57.818419 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.818346 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hjssk" event={"ID":"09ec7040-20c0-44ef-9d2c-b717f2b6dd0b","Type":"ContainerStarted","Data":"8923ab885717bc2c6f915cc0c206a647f3303f23b6fbf7a476d0ecd18ded3764"} Jul 06 16:51:57.882396 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.882331 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb4qk\" (UniqueName: \"kubernetes.io/projected/89d3d741-e1fa-4702-a68a-0e7454c3e791-kube-api-access-tb4qk\") pod \"authorino-75d556bd77-47p5k\" (UID: \"89d3d741-e1fa-4702-a68a-0e7454c3e791\") " pod="kuadrant-system/authorino-75d556bd77-47p5k" Jul 06 16:51:57.892758 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:57.892732 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb4qk\" (UniqueName: \"kubernetes.io/projected/89d3d741-e1fa-4702-a68a-0e7454c3e791-kube-api-access-tb4qk\") pod \"authorino-75d556bd77-47p5k\" (UID: \"89d3d741-e1fa-4702-a68a-0e7454c3e791\") " pod="kuadrant-system/authorino-75d556bd77-47p5k" Jul 06 16:51:58.019780 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:58.019756 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-47p5k" Jul 06 16:51:58.156766 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:58.156738 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-47p5k"] Jul 06 16:51:58.157682 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:51:58.157657 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89d3d741_e1fa_4702_a68a_0e7454c3e791.slice/crio-fce0b2b26c765069bc884813229e91ffd4c47179b7ef45a20801d27052655c70 WatchSource:0}: Error finding container fce0b2b26c765069bc884813229e91ffd4c47179b7ef45a20801d27052655c70: Status 404 returned error can't find the container with id fce0b2b26c765069bc884813229e91ffd4c47179b7ef45a20801d27052655c70 Jul 06 16:51:58.830205 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:51:58.830163 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-47p5k" event={"ID":"89d3d741-e1fa-4702-a68a-0e7454c3e791","Type":"ContainerStarted","Data":"fce0b2b26c765069bc884813229e91ffd4c47179b7ef45a20801d27052655c70"} Jul 06 16:52:01.859801 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:01.859764 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-47p5k" event={"ID":"89d3d741-e1fa-4702-a68a-0e7454c3e791","Type":"ContainerStarted","Data":"d356f1fb0ff848713a3eae59603ec0e1f2d2bef5aa4e354d074be3e403111b09"} Jul 06 16:52:01.861371 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:01.861340 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hjssk" event={"ID":"09ec7040-20c0-44ef-9d2c-b717f2b6dd0b","Type":"ContainerStarted","Data":"b9b9caf98177bc5a45ef6f58447b302410a31998b7495a4bd690ae48466963a8"} Jul 06 16:52:01.877636 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:01.877595 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-47p5k" podStartSLOduration=1.360608412 podStartE2EDuration="4.87758027s" totalImagesPullingTime="3.516971858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:51:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:52:01.875238823 +0000 UTC m=+836.053097949" watchObservedRunningTime="2026-07-06 16:52:01.87758027 +0000 UTC m=+836.055439381" Jul 06 16:52:01.891927 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:01.891882 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-hjssk" podStartSLOduration=1.019546425 podStartE2EDuration="4.891871179s" totalImagesPullingTime="3.872324754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:51:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:52:01.890561026 +0000 UTC m=+836.068420139" watchObservedRunningTime="2026-07-06 16:52:01.891871179 +0000 UTC m=+836.069730344" Jul 06 16:52:01.919525 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:01.919440 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-hjssk"] Jul 06 16:52:03.871688 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:03.871650 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-hjssk" podUID="09ec7040-20c0-44ef-9d2c-b717f2b6dd0b" containerName="authorino" containerID="cri-o://b9b9caf98177bc5a45ef6f58447b302410a31998b7495a4bd690ae48466963a8" gracePeriod=30 Jul 06 16:52:04.122386 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:04.122334 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hjssk" Jul 06 16:52:04.237036 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:04.237001 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hw8jr\" (UniqueName: \"kubernetes.io/projected/09ec7040-20c0-44ef-9d2c-b717f2b6dd0b-kube-api-access-hw8jr\") pod \"09ec7040-20c0-44ef-9d2c-b717f2b6dd0b\" (UID: \"09ec7040-20c0-44ef-9d2c-b717f2b6dd0b\") " Jul 06 16:52:04.238972 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:04.238947 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09ec7040-20c0-44ef-9d2c-b717f2b6dd0b-kube-api-access-hw8jr" (OuterVolumeSpecName: "kube-api-access-hw8jr") pod "09ec7040-20c0-44ef-9d2c-b717f2b6dd0b" (UID: "09ec7040-20c0-44ef-9d2c-b717f2b6dd0b"). InnerVolumeSpecName "kube-api-access-hw8jr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:52:04.338126 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:04.338095 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hw8jr\" (UniqueName: \"kubernetes.io/projected/09ec7040-20c0-44ef-9d2c-b717f2b6dd0b-kube-api-access-hw8jr\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:52:04.878063 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:04.878025 2580 generic.go:356] "Generic (PLEG): container finished" podID="09ec7040-20c0-44ef-9d2c-b717f2b6dd0b" containerID="b9b9caf98177bc5a45ef6f58447b302410a31998b7495a4bd690ae48466963a8" exitCode=0 Jul 06 16:52:04.878525 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:04.878070 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hjssk" event={"ID":"09ec7040-20c0-44ef-9d2c-b717f2b6dd0b","Type":"ContainerDied","Data":"b9b9caf98177bc5a45ef6f58447b302410a31998b7495a4bd690ae48466963a8"} Jul 06 16:52:04.878525 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:04.878087 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hjssk" Jul 06 16:52:04.878525 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:04.878107 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hjssk" event={"ID":"09ec7040-20c0-44ef-9d2c-b717f2b6dd0b","Type":"ContainerDied","Data":"8923ab885717bc2c6f915cc0c206a647f3303f23b6fbf7a476d0ecd18ded3764"} Jul 06 16:52:04.878525 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:04.878128 2580 scope.go:117] "RemoveContainer" containerID="b9b9caf98177bc5a45ef6f58447b302410a31998b7495a4bd690ae48466963a8" Jul 06 16:52:04.887365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:04.887343 2580 scope.go:117] "RemoveContainer" containerID="b9b9caf98177bc5a45ef6f58447b302410a31998b7495a4bd690ae48466963a8" Jul 06 16:52:04.887607 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:52:04.887588 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9b9caf98177bc5a45ef6f58447b302410a31998b7495a4bd690ae48466963a8\": container with ID starting with b9b9caf98177bc5a45ef6f58447b302410a31998b7495a4bd690ae48466963a8 not found: ID does not exist" containerID="b9b9caf98177bc5a45ef6f58447b302410a31998b7495a4bd690ae48466963a8" Jul 06 16:52:04.887665 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:04.887616 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9b9caf98177bc5a45ef6f58447b302410a31998b7495a4bd690ae48466963a8"} err="failed to get container status \"b9b9caf98177bc5a45ef6f58447b302410a31998b7495a4bd690ae48466963a8\": rpc error: code = NotFound desc = could not find container \"b9b9caf98177bc5a45ef6f58447b302410a31998b7495a4bd690ae48466963a8\": container with ID starting with b9b9caf98177bc5a45ef6f58447b302410a31998b7495a4bd690ae48466963a8 not found: ID does not exist" Jul 06 16:52:04.897136 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:04.897114 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-hjssk"] Jul 06 16:52:04.903891 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:04.903869 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-hjssk"] Jul 06 16:52:06.264738 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:06.264709 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09ec7040-20c0-44ef-9d2c-b717f2b6dd0b" path="/var/lib/kubelet/pods/09ec7040-20c0-44ef-9d2c-b717f2b6dd0b/volumes" Jul 06 16:52:30.850635 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:30.850601 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fmr9j"] Jul 06 16:52:30.851091 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:30.850966 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09ec7040-20c0-44ef-9d2c-b717f2b6dd0b" containerName="authorino" Jul 06 16:52:30.851091 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:30.850979 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="09ec7040-20c0-44ef-9d2c-b717f2b6dd0b" containerName="authorino" Jul 06 16:52:30.851091 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:30.851049 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="09ec7040-20c0-44ef-9d2c-b717f2b6dd0b" containerName="authorino" Jul 06 16:52:30.855308 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:30.855290 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fmr9j" Jul 06 16:52:30.863997 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:30.863973 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fmr9j"] Jul 06 16:52:30.953883 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:30.953855 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7gcc\" (UniqueName: \"kubernetes.io/projected/933b4398-ad5d-433a-8bc1-d4887bc47ae7-kube-api-access-k7gcc\") pod \"authorino-5d4b67c446-fmr9j\" (UID: \"933b4398-ad5d-433a-8bc1-d4887bc47ae7\") " pod="kuadrant-system/authorino-5d4b67c446-fmr9j" Jul 06 16:52:31.054492 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.054429 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7gcc\" (UniqueName: \"kubernetes.io/projected/933b4398-ad5d-433a-8bc1-d4887bc47ae7-kube-api-access-k7gcc\") pod \"authorino-5d4b67c446-fmr9j\" (UID: \"933b4398-ad5d-433a-8bc1-d4887bc47ae7\") " pod="kuadrant-system/authorino-5d4b67c446-fmr9j" Jul 06 16:52:31.064289 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.064265 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7gcc\" (UniqueName: \"kubernetes.io/projected/933b4398-ad5d-433a-8bc1-d4887bc47ae7-kube-api-access-k7gcc\") pod \"authorino-5d4b67c446-fmr9j\" (UID: \"933b4398-ad5d-433a-8bc1-d4887bc47ae7\") " pod="kuadrant-system/authorino-5d4b67c446-fmr9j" Jul 06 16:52:31.129132 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.129054 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fmr9j"] Jul 06 16:52:31.129312 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.129301 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fmr9j" Jul 06 16:52:31.161572 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.161541 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-977899f96-7p427"] Jul 06 16:52:31.165357 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.165340 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-977899f96-7p427" Jul 06 16:52:31.176689 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.176664 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-977899f96-7p427"] Jul 06 16:52:31.256048 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.256018 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dbs8\" (UniqueName: \"kubernetes.io/projected/b3f12d41-ecd5-41d1-b126-60960ce9ebd1-kube-api-access-4dbs8\") pod \"authorino-977899f96-7p427\" (UID: \"b3f12d41-ecd5-41d1-b126-60960ce9ebd1\") " pod="kuadrant-system/authorino-977899f96-7p427" Jul 06 16:52:31.269740 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.269719 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fmr9j"] Jul 06 16:52:31.271773 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:52:31.271749 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod933b4398_ad5d_433a_8bc1_d4887bc47ae7.slice/crio-e69d937d6839b42fadce8f3ccb187b0f9977b3b704834be536e5ed362515dbfc WatchSource:0}: Error finding container e69d937d6839b42fadce8f3ccb187b0f9977b3b704834be536e5ed362515dbfc: Status 404 returned error can't find the container with id e69d937d6839b42fadce8f3ccb187b0f9977b3b704834be536e5ed362515dbfc Jul 06 16:52:31.357199 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.356931 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dbs8\" (UniqueName: \"kubernetes.io/projected/b3f12d41-ecd5-41d1-b126-60960ce9ebd1-kube-api-access-4dbs8\") pod \"authorino-977899f96-7p427\" (UID: \"b3f12d41-ecd5-41d1-b126-60960ce9ebd1\") " pod="kuadrant-system/authorino-977899f96-7p427" Jul 06 16:52:31.369793 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.369767 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dbs8\" (UniqueName: \"kubernetes.io/projected/b3f12d41-ecd5-41d1-b126-60960ce9ebd1-kube-api-access-4dbs8\") pod \"authorino-977899f96-7p427\" (UID: \"b3f12d41-ecd5-41d1-b126-60960ce9ebd1\") " pod="kuadrant-system/authorino-977899f96-7p427" Jul 06 16:52:31.461612 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.461549 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-977899f96-7p427"] Jul 06 16:52:31.461740 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.461729 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-977899f96-7p427" Jul 06 16:52:31.488578 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.488546 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75b9554c88-dx6v9"] Jul 06 16:52:31.492165 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.492146 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75b9554c88-dx6v9" Jul 06 16:52:31.494583 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.494558 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 06 16:52:31.516240 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.516209 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75b9554c88-dx6v9"] Jul 06 16:52:31.558115 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.558085 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hnvn\" (UniqueName: \"kubernetes.io/projected/fd6d1862-61aa-41b6-a0ab-64914a1b8cd5-kube-api-access-8hnvn\") pod \"authorino-75b9554c88-dx6v9\" (UID: \"fd6d1862-61aa-41b6-a0ab-64914a1b8cd5\") " pod="kuadrant-system/authorino-75b9554c88-dx6v9" Jul 06 16:52:31.558244 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.558136 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fd6d1862-61aa-41b6-a0ab-64914a1b8cd5-tls-cert\") pod \"authorino-75b9554c88-dx6v9\" (UID: \"fd6d1862-61aa-41b6-a0ab-64914a1b8cd5\") " pod="kuadrant-system/authorino-75b9554c88-dx6v9" Jul 06 16:52:31.606068 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.606044 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-977899f96-7p427"] Jul 06 16:52:31.608043 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:52:31.608016 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3f12d41_ecd5_41d1_b126_60960ce9ebd1.slice/crio-dc67b87390a7e01176592a469975dfcd61732061c78142f0913d5d24f787df83 WatchSource:0}: Error finding container dc67b87390a7e01176592a469975dfcd61732061c78142f0913d5d24f787df83: Status 404 returned error can't find the container with id dc67b87390a7e01176592a469975dfcd61732061c78142f0913d5d24f787df83 Jul 06 16:52:31.659149 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.659122 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hnvn\" (UniqueName: \"kubernetes.io/projected/fd6d1862-61aa-41b6-a0ab-64914a1b8cd5-kube-api-access-8hnvn\") pod \"authorino-75b9554c88-dx6v9\" (UID: \"fd6d1862-61aa-41b6-a0ab-64914a1b8cd5\") " pod="kuadrant-system/authorino-75b9554c88-dx6v9" Jul 06 16:52:31.659259 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.659160 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fd6d1862-61aa-41b6-a0ab-64914a1b8cd5-tls-cert\") pod \"authorino-75b9554c88-dx6v9\" (UID: \"fd6d1862-61aa-41b6-a0ab-64914a1b8cd5\") " pod="kuadrant-system/authorino-75b9554c88-dx6v9" Jul 06 16:52:31.661837 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.661811 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fd6d1862-61aa-41b6-a0ab-64914a1b8cd5-tls-cert\") pod \"authorino-75b9554c88-dx6v9\" (UID: \"fd6d1862-61aa-41b6-a0ab-64914a1b8cd5\") " pod="kuadrant-system/authorino-75b9554c88-dx6v9" Jul 06 16:52:31.679113 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.679088 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hnvn\" (UniqueName: \"kubernetes.io/projected/fd6d1862-61aa-41b6-a0ab-64914a1b8cd5-kube-api-access-8hnvn\") pod \"authorino-75b9554c88-dx6v9\" (UID: \"fd6d1862-61aa-41b6-a0ab-64914a1b8cd5\") " pod="kuadrant-system/authorino-75b9554c88-dx6v9" Jul 06 16:52:31.801732 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.801633 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75b9554c88-dx6v9" Jul 06 16:52:31.938260 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:31.938230 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75b9554c88-dx6v9"] Jul 06 16:52:31.938655 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:52:31.938538 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd6d1862_61aa_41b6_a0ab_64914a1b8cd5.slice/crio-2018033b5426321e0faae615e02fa5d9342d8698dac35f7c46c45e24762c7825 WatchSource:0}: Error finding container 2018033b5426321e0faae615e02fa5d9342d8698dac35f7c46c45e24762c7825: Status 404 returned error can't find the container with id 2018033b5426321e0faae615e02fa5d9342d8698dac35f7c46c45e24762c7825 Jul 06 16:52:32.034189 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:32.033891 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-977899f96-7p427" podUID="b3f12d41-ecd5-41d1-b126-60960ce9ebd1" containerName="authorino" containerID="cri-o://a3b3d7100d1f670037abbb5e31536fb2c8f4c068764b9e0db3afe32132cfc857" gracePeriod=30 Jul 06 16:52:32.034189 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:32.034172 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-977899f96-7p427" event={"ID":"b3f12d41-ecd5-41d1-b126-60960ce9ebd1","Type":"ContainerStarted","Data":"a3b3d7100d1f670037abbb5e31536fb2c8f4c068764b9e0db3afe32132cfc857"} Jul 06 16:52:32.034417 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:32.034212 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-977899f96-7p427" event={"ID":"b3f12d41-ecd5-41d1-b126-60960ce9ebd1","Type":"ContainerStarted","Data":"dc67b87390a7e01176592a469975dfcd61732061c78142f0913d5d24f787df83"} Jul 06 16:52:32.036039 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:32.036005 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fmr9j" event={"ID":"933b4398-ad5d-433a-8bc1-d4887bc47ae7","Type":"ContainerStarted","Data":"459bc630e7544a24c749edebf0e4f69fece20b707d99344671ccb6a6503fbb99"} Jul 06 16:52:32.036170 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:32.036046 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fmr9j" event={"ID":"933b4398-ad5d-433a-8bc1-d4887bc47ae7","Type":"ContainerStarted","Data":"e69d937d6839b42fadce8f3ccb187b0f9977b3b704834be536e5ed362515dbfc"} Jul 06 16:52:32.036170 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:32.036052 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-fmr9j" podUID="933b4398-ad5d-433a-8bc1-d4887bc47ae7" containerName="authorino" containerID="cri-o://459bc630e7544a24c749edebf0e4f69fece20b707d99344671ccb6a6503fbb99" gracePeriod=30 Jul 06 16:52:32.037602 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:32.037581 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75b9554c88-dx6v9" event={"ID":"fd6d1862-61aa-41b6-a0ab-64914a1b8cd5","Type":"ContainerStarted","Data":"2018033b5426321e0faae615e02fa5d9342d8698dac35f7c46c45e24762c7825"} Jul 06 16:52:32.053978 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:32.053911 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-977899f96-7p427" podStartSLOduration=0.710403283 podStartE2EDuration="1.053898199s" totalImagesPullingTime="343.494916ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:52:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:52:32.053106916 +0000 UTC m=+866.230966030" watchObservedRunningTime="2026-07-06 16:52:32.053898199 +0000 UTC m=+866.231757311" Jul 06 16:52:32.077516 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:32.077456 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-fmr9j" podStartSLOduration=1.69015534 podStartE2EDuration="2.077441686s" totalImagesPullingTime="387.286346ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:52:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:52:32.075503568 +0000 UTC m=+866.253362682" watchObservedRunningTime="2026-07-06 16:52:32.077441686 +0000 UTC m=+866.255300837" Jul 06 16:52:32.308355 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:32.308287 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fmr9j" Jul 06 16:52:32.311080 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:32.311057 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-977899f96-7p427" Jul 06 16:52:32.364646 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:32.364620 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7gcc\" (UniqueName: \"kubernetes.io/projected/933b4398-ad5d-433a-8bc1-d4887bc47ae7-kube-api-access-k7gcc\") pod \"933b4398-ad5d-433a-8bc1-d4887bc47ae7\" (UID: \"933b4398-ad5d-433a-8bc1-d4887bc47ae7\") " Jul 06 16:52:32.364949 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:32.364663 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dbs8\" (UniqueName: \"kubernetes.io/projected/b3f12d41-ecd5-41d1-b126-60960ce9ebd1-kube-api-access-4dbs8\") pod \"b3f12d41-ecd5-41d1-b126-60960ce9ebd1\" (UID: \"b3f12d41-ecd5-41d1-b126-60960ce9ebd1\") " Jul 06 16:52:32.366899 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:32.366758 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/933b4398-ad5d-433a-8bc1-d4887bc47ae7-kube-api-access-k7gcc" (OuterVolumeSpecName: "kube-api-access-k7gcc") pod "933b4398-ad5d-433a-8bc1-d4887bc47ae7" (UID: "933b4398-ad5d-433a-8bc1-d4887bc47ae7"). InnerVolumeSpecName "kube-api-access-k7gcc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:52:32.367274 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:32.367229 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3f12d41-ecd5-41d1-b126-60960ce9ebd1-kube-api-access-4dbs8" (OuterVolumeSpecName: "kube-api-access-4dbs8") pod "b3f12d41-ecd5-41d1-b126-60960ce9ebd1" (UID: "b3f12d41-ecd5-41d1-b126-60960ce9ebd1"). InnerVolumeSpecName "kube-api-access-4dbs8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:52:32.465821 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:32.465772 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7gcc\" (UniqueName: \"kubernetes.io/projected/933b4398-ad5d-433a-8bc1-d4887bc47ae7-kube-api-access-k7gcc\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:52:32.465821 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:32.465801 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dbs8\" (UniqueName: \"kubernetes.io/projected/b3f12d41-ecd5-41d1-b126-60960ce9ebd1-kube-api-access-4dbs8\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:52:33.043458 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.043423 2580 generic.go:356] "Generic (PLEG): container finished" podID="933b4398-ad5d-433a-8bc1-d4887bc47ae7" containerID="459bc630e7544a24c749edebf0e4f69fece20b707d99344671ccb6a6503fbb99" exitCode=0 Jul 06 16:52:33.043896 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.043485 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-fmr9j" Jul 06 16:52:33.043896 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.043514 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fmr9j" event={"ID":"933b4398-ad5d-433a-8bc1-d4887bc47ae7","Type":"ContainerDied","Data":"459bc630e7544a24c749edebf0e4f69fece20b707d99344671ccb6a6503fbb99"} Jul 06 16:52:33.043896 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.043553 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-fmr9j" event={"ID":"933b4398-ad5d-433a-8bc1-d4887bc47ae7","Type":"ContainerDied","Data":"e69d937d6839b42fadce8f3ccb187b0f9977b3b704834be536e5ed362515dbfc"} Jul 06 16:52:33.043896 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.043570 2580 scope.go:117] "RemoveContainer" containerID="459bc630e7544a24c749edebf0e4f69fece20b707d99344671ccb6a6503fbb99" Jul 06 16:52:33.045157 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.045134 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75b9554c88-dx6v9" event={"ID":"fd6d1862-61aa-41b6-a0ab-64914a1b8cd5","Type":"ContainerStarted","Data":"6c37595469f83621fb0d2911636cd110c44fd634472db2fe2a018a2d143ff536"} Jul 06 16:52:33.046399 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.046376 2580 generic.go:356] "Generic (PLEG): container finished" podID="b3f12d41-ecd5-41d1-b126-60960ce9ebd1" containerID="a3b3d7100d1f670037abbb5e31536fb2c8f4c068764b9e0db3afe32132cfc857" exitCode=2 Jul 06 16:52:33.046550 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.046404 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-977899f96-7p427" Jul 06 16:52:33.046550 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.046427 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-977899f96-7p427" event={"ID":"b3f12d41-ecd5-41d1-b126-60960ce9ebd1","Type":"ContainerDied","Data":"a3b3d7100d1f670037abbb5e31536fb2c8f4c068764b9e0db3afe32132cfc857"} Jul 06 16:52:33.046550 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.046451 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-977899f96-7p427" event={"ID":"b3f12d41-ecd5-41d1-b126-60960ce9ebd1","Type":"ContainerDied","Data":"dc67b87390a7e01176592a469975dfcd61732061c78142f0913d5d24f787df83"} Jul 06 16:52:33.053044 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.053024 2580 scope.go:117] "RemoveContainer" containerID="459bc630e7544a24c749edebf0e4f69fece20b707d99344671ccb6a6503fbb99" Jul 06 16:52:33.053302 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:52:33.053282 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"459bc630e7544a24c749edebf0e4f69fece20b707d99344671ccb6a6503fbb99\": container with ID starting with 459bc630e7544a24c749edebf0e4f69fece20b707d99344671ccb6a6503fbb99 not found: ID does not exist" containerID="459bc630e7544a24c749edebf0e4f69fece20b707d99344671ccb6a6503fbb99" Jul 06 16:52:33.053372 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.053314 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"459bc630e7544a24c749edebf0e4f69fece20b707d99344671ccb6a6503fbb99"} err="failed to get container status \"459bc630e7544a24c749edebf0e4f69fece20b707d99344671ccb6a6503fbb99\": rpc error: code = NotFound desc = could not find container \"459bc630e7544a24c749edebf0e4f69fece20b707d99344671ccb6a6503fbb99\": container with ID starting with 459bc630e7544a24c749edebf0e4f69fece20b707d99344671ccb6a6503fbb99 not found: ID does not exist" Jul 06 16:52:33.053372 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.053338 2580 scope.go:117] "RemoveContainer" containerID="a3b3d7100d1f670037abbb5e31536fb2c8f4c068764b9e0db3afe32132cfc857" Jul 06 16:52:33.061227 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.061208 2580 scope.go:117] "RemoveContainer" containerID="a3b3d7100d1f670037abbb5e31536fb2c8f4c068764b9e0db3afe32132cfc857" Jul 06 16:52:33.061541 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:52:33.061462 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3b3d7100d1f670037abbb5e31536fb2c8f4c068764b9e0db3afe32132cfc857\": container with ID starting with a3b3d7100d1f670037abbb5e31536fb2c8f4c068764b9e0db3afe32132cfc857 not found: ID does not exist" containerID="a3b3d7100d1f670037abbb5e31536fb2c8f4c068764b9e0db3afe32132cfc857" Jul 06 16:52:33.061541 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.061509 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3b3d7100d1f670037abbb5e31536fb2c8f4c068764b9e0db3afe32132cfc857"} err="failed to get container status \"a3b3d7100d1f670037abbb5e31536fb2c8f4c068764b9e0db3afe32132cfc857\": rpc error: code = NotFound desc = could not find container \"a3b3d7100d1f670037abbb5e31536fb2c8f4c068764b9e0db3afe32132cfc857\": container with ID starting with a3b3d7100d1f670037abbb5e31536fb2c8f4c068764b9e0db3afe32132cfc857 not found: ID does not exist" Jul 06 16:52:33.067516 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.067461 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75b9554c88-dx6v9" podStartSLOduration=1.674107091 podStartE2EDuration="2.06744977s" totalImagesPullingTime="393.342679ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:52:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:52:33.065314535 +0000 UTC m=+867.243173646" watchObservedRunningTime="2026-07-06 16:52:33.06744977 +0000 UTC m=+867.245308883" Jul 06 16:52:33.079667 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.079642 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-977899f96-7p427"] Jul 06 16:52:33.084827 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.084800 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-977899f96-7p427"] Jul 06 16:52:33.095066 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.094816 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-47p5k"] Jul 06 16:52:33.095066 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.094993 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-47p5k" podUID="89d3d741-e1fa-4702-a68a-0e7454c3e791" containerName="authorino" containerID="cri-o://d356f1fb0ff848713a3eae59603ec0e1f2d2bef5aa4e354d074be3e403111b09" gracePeriod=30 Jul 06 16:52:33.097865 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.097776 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fmr9j"] Jul 06 16:52:33.104153 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.104132 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-fmr9j"] Jul 06 16:52:33.342842 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.342819 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-47p5k" Jul 06 16:52:33.472529 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.472495 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tb4qk\" (UniqueName: \"kubernetes.io/projected/89d3d741-e1fa-4702-a68a-0e7454c3e791-kube-api-access-tb4qk\") pod \"89d3d741-e1fa-4702-a68a-0e7454c3e791\" (UID: \"89d3d741-e1fa-4702-a68a-0e7454c3e791\") " Jul 06 16:52:33.474528 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.474500 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89d3d741-e1fa-4702-a68a-0e7454c3e791-kube-api-access-tb4qk" (OuterVolumeSpecName: "kube-api-access-tb4qk") pod "89d3d741-e1fa-4702-a68a-0e7454c3e791" (UID: "89d3d741-e1fa-4702-a68a-0e7454c3e791"). InnerVolumeSpecName "kube-api-access-tb4qk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:52:33.573578 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:33.573546 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tb4qk\" (UniqueName: \"kubernetes.io/projected/89d3d741-e1fa-4702-a68a-0e7454c3e791-kube-api-access-tb4qk\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:52:34.052783 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:34.052743 2580 generic.go:356] "Generic (PLEG): container finished" podID="89d3d741-e1fa-4702-a68a-0e7454c3e791" containerID="d356f1fb0ff848713a3eae59603ec0e1f2d2bef5aa4e354d074be3e403111b09" exitCode=0 Jul 06 16:52:34.052783 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:34.052790 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-47p5k" Jul 06 16:52:34.053232 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:34.052830 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-47p5k" event={"ID":"89d3d741-e1fa-4702-a68a-0e7454c3e791","Type":"ContainerDied","Data":"d356f1fb0ff848713a3eae59603ec0e1f2d2bef5aa4e354d074be3e403111b09"} Jul 06 16:52:34.053232 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:34.052882 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-47p5k" event={"ID":"89d3d741-e1fa-4702-a68a-0e7454c3e791","Type":"ContainerDied","Data":"fce0b2b26c765069bc884813229e91ffd4c47179b7ef45a20801d27052655c70"} Jul 06 16:52:34.053232 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:34.052910 2580 scope.go:117] "RemoveContainer" containerID="d356f1fb0ff848713a3eae59603ec0e1f2d2bef5aa4e354d074be3e403111b09" Jul 06 16:52:34.062218 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:34.062202 2580 scope.go:117] "RemoveContainer" containerID="d356f1fb0ff848713a3eae59603ec0e1f2d2bef5aa4e354d074be3e403111b09" Jul 06 16:52:34.062445 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:52:34.062429 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d356f1fb0ff848713a3eae59603ec0e1f2d2bef5aa4e354d074be3e403111b09\": container with ID starting with d356f1fb0ff848713a3eae59603ec0e1f2d2bef5aa4e354d074be3e403111b09 not found: ID does not exist" containerID="d356f1fb0ff848713a3eae59603ec0e1f2d2bef5aa4e354d074be3e403111b09" Jul 06 16:52:34.062546 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:34.062452 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d356f1fb0ff848713a3eae59603ec0e1f2d2bef5aa4e354d074be3e403111b09"} err="failed to get container status \"d356f1fb0ff848713a3eae59603ec0e1f2d2bef5aa4e354d074be3e403111b09\": rpc error: code = NotFound desc = could not find container \"d356f1fb0ff848713a3eae59603ec0e1f2d2bef5aa4e354d074be3e403111b09\": container with ID starting with d356f1fb0ff848713a3eae59603ec0e1f2d2bef5aa4e354d074be3e403111b09 not found: ID does not exist" Jul 06 16:52:34.076042 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:34.076018 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-47p5k"] Jul 06 16:52:34.078045 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:34.078027 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-47p5k"] Jul 06 16:52:34.264366 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:34.264336 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89d3d741-e1fa-4702-a68a-0e7454c3e791" path="/var/lib/kubelet/pods/89d3d741-e1fa-4702-a68a-0e7454c3e791/volumes" Jul 06 16:52:34.264704 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:34.264688 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="933b4398-ad5d-433a-8bc1-d4887bc47ae7" path="/var/lib/kubelet/pods/933b4398-ad5d-433a-8bc1-d4887bc47ae7/volumes" Jul 06 16:52:34.264980 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:52:34.264967 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3f12d41-ecd5-41d1-b126-60960ce9ebd1" path="/var/lib/kubelet/pods/b3f12d41-ecd5-41d1-b126-60960ce9ebd1/volumes" Jul 06 16:53:06.238975 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:06.238942 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 16:53:06.239916 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:06.239896 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 16:53:06.247182 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:06.247156 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 16:53:06.248044 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:06.248025 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 16:53:33.303292 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.303259 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns"] Jul 06 16:53:33.303736 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.303627 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89d3d741-e1fa-4702-a68a-0e7454c3e791" containerName="authorino" Jul 06 16:53:33.303736 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.303638 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="89d3d741-e1fa-4702-a68a-0e7454c3e791" containerName="authorino" Jul 06 16:53:33.303736 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.303657 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="933b4398-ad5d-433a-8bc1-d4887bc47ae7" containerName="authorino" Jul 06 16:53:33.303736 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.303662 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="933b4398-ad5d-433a-8bc1-d4887bc47ae7" containerName="authorino" Jul 06 16:53:33.303736 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.303674 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3f12d41-ecd5-41d1-b126-60960ce9ebd1" containerName="authorino" Jul 06 16:53:33.303736 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.303679 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3f12d41-ecd5-41d1-b126-60960ce9ebd1" containerName="authorino" Jul 06 16:53:33.303736 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.303732 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="933b4398-ad5d-433a-8bc1-d4887bc47ae7" containerName="authorino" Jul 06 16:53:33.303736 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.303740 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3f12d41-ecd5-41d1-b126-60960ce9ebd1" containerName="authorino" Jul 06 16:53:33.303983 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.303750 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="89d3d741-e1fa-4702-a68a-0e7454c3e791" containerName="authorino" Jul 06 16:53:33.306680 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.306664 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:33.308965 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.308947 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 16:53:33.309368 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.309350 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 16:53:33.309891 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.309877 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 06 16:53:33.309962 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.309944 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-s9bmb\"" Jul 06 16:53:33.317339 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.317317 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns"] Jul 06 16:53:33.371311 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.371279 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8d2b9b9-75ac-4297-9207-de2cd1dc876d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns\" (UID: \"a8d2b9b9-75ac-4297-9207-de2cd1dc876d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:33.371443 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.371330 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5rb6\" (UniqueName: \"kubernetes.io/projected/a8d2b9b9-75ac-4297-9207-de2cd1dc876d-kube-api-access-s5rb6\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns\" (UID: \"a8d2b9b9-75ac-4297-9207-de2cd1dc876d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:33.371443 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.371394 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8d2b9b9-75ac-4297-9207-de2cd1dc876d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns\" (UID: \"a8d2b9b9-75ac-4297-9207-de2cd1dc876d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:33.371527 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.371458 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8d2b9b9-75ac-4297-9207-de2cd1dc876d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns\" (UID: \"a8d2b9b9-75ac-4297-9207-de2cd1dc876d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:33.371527 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.371497 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8d2b9b9-75ac-4297-9207-de2cd1dc876d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns\" (UID: \"a8d2b9b9-75ac-4297-9207-de2cd1dc876d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:33.371594 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.371529 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8d2b9b9-75ac-4297-9207-de2cd1dc876d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns\" (UID: \"a8d2b9b9-75ac-4297-9207-de2cd1dc876d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:33.472008 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.471969 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5rb6\" (UniqueName: \"kubernetes.io/projected/a8d2b9b9-75ac-4297-9207-de2cd1dc876d-kube-api-access-s5rb6\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns\" (UID: \"a8d2b9b9-75ac-4297-9207-de2cd1dc876d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:33.472163 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.472029 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8d2b9b9-75ac-4297-9207-de2cd1dc876d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns\" (UID: \"a8d2b9b9-75ac-4297-9207-de2cd1dc876d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:33.472163 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.472058 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8d2b9b9-75ac-4297-9207-de2cd1dc876d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns\" (UID: \"a8d2b9b9-75ac-4297-9207-de2cd1dc876d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:33.472285 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.472200 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8d2b9b9-75ac-4297-9207-de2cd1dc876d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns\" (UID: \"a8d2b9b9-75ac-4297-9207-de2cd1dc876d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:33.472285 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.472243 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8d2b9b9-75ac-4297-9207-de2cd1dc876d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns\" (UID: \"a8d2b9b9-75ac-4297-9207-de2cd1dc876d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:33.472393 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.472307 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8d2b9b9-75ac-4297-9207-de2cd1dc876d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns\" (UID: \"a8d2b9b9-75ac-4297-9207-de2cd1dc876d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:33.472451 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.472399 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8d2b9b9-75ac-4297-9207-de2cd1dc876d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns\" (UID: \"a8d2b9b9-75ac-4297-9207-de2cd1dc876d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:33.472669 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.472647 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8d2b9b9-75ac-4297-9207-de2cd1dc876d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns\" (UID: \"a8d2b9b9-75ac-4297-9207-de2cd1dc876d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:33.472669 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.472654 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8d2b9b9-75ac-4297-9207-de2cd1dc876d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns\" (UID: \"a8d2b9b9-75ac-4297-9207-de2cd1dc876d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:33.474419 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.474388 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8d2b9b9-75ac-4297-9207-de2cd1dc876d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns\" (UID: \"a8d2b9b9-75ac-4297-9207-de2cd1dc876d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:33.474653 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.474633 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8d2b9b9-75ac-4297-9207-de2cd1dc876d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns\" (UID: \"a8d2b9b9-75ac-4297-9207-de2cd1dc876d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:33.480951 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.480930 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5rb6\" (UniqueName: \"kubernetes.io/projected/a8d2b9b9-75ac-4297-9207-de2cd1dc876d-kube-api-access-s5rb6\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns\" (UID: \"a8d2b9b9-75ac-4297-9207-de2cd1dc876d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:33.616872 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.616840 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:33.751637 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.751614 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns"] Jul 06 16:53:33.753675 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:53:33.753650 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8d2b9b9_75ac_4297_9207_de2cd1dc876d.slice/crio-b8455185314615b5393a44bb069f28891bedc0cf1cc685b5b9e81aedb5197ec0 WatchSource:0}: Error finding container b8455185314615b5393a44bb069f28891bedc0cf1cc685b5b9e81aedb5197ec0: Status 404 returned error can't find the container with id b8455185314615b5393a44bb069f28891bedc0cf1cc685b5b9e81aedb5197ec0 Jul 06 16:53:33.755681 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:33.755662 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 16:53:34.367759 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:34.367718 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" event={"ID":"a8d2b9b9-75ac-4297-9207-de2cd1dc876d","Type":"ContainerStarted","Data":"b8455185314615b5393a44bb069f28891bedc0cf1cc685b5b9e81aedb5197ec0"} Jul 06 16:53:35.374841 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:35.374805 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" event={"ID":"a8d2b9b9-75ac-4297-9207-de2cd1dc876d","Type":"ContainerStarted","Data":"25eb1dc8ff7c36e3815fcdc8d420c56ee665e7014bd2535689e4d92dd4bf38e2"} Jul 06 16:53:35.397043 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:35.396983 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" podStartSLOduration=1.31201538 podStartE2EDuration="2.396966171s" totalImagesPullingTime="1.084950791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:53:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:53:35.393144709 +0000 UTC m=+929.571003820" watchObservedRunningTime="2026-07-06 16:53:35.396966171 +0000 UTC m=+929.574825286" Jul 06 16:53:43.617648 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:43.617603 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:43.618355 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:43.617670 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:43.630148 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:43.630122 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:53:44.432641 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:53:44.432615 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns" Jul 06 16:54:20.929631 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:20.929599 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9"] Jul 06 16:54:20.933007 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:20.932988 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:20.936178 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:20.936151 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 06 16:54:20.960096 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:20.960073 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9"] Jul 06 16:54:20.985824 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:20.985793 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m6bk\" (UniqueName: \"kubernetes.io/projected/b101708b-1016-436a-8a7c-d14b010767ba-kube-api-access-6m6bk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kzr9\" (UID: \"b101708b-1016-436a-8a7c-d14b010767ba\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:20.985929 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:20.985830 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b101708b-1016-436a-8a7c-d14b010767ba-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kzr9\" (UID: \"b101708b-1016-436a-8a7c-d14b010767ba\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:20.986742 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:20.985935 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b101708b-1016-436a-8a7c-d14b010767ba-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kzr9\" (UID: \"b101708b-1016-436a-8a7c-d14b010767ba\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:20.986742 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:20.985960 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b101708b-1016-436a-8a7c-d14b010767ba-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kzr9\" (UID: \"b101708b-1016-436a-8a7c-d14b010767ba\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:20.986742 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:20.986012 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b101708b-1016-436a-8a7c-d14b010767ba-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kzr9\" (UID: \"b101708b-1016-436a-8a7c-d14b010767ba\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:20.986742 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:20.986055 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b101708b-1016-436a-8a7c-d14b010767ba-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kzr9\" (UID: \"b101708b-1016-436a-8a7c-d14b010767ba\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:21.087002 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:21.086972 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b101708b-1016-436a-8a7c-d14b010767ba-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kzr9\" (UID: \"b101708b-1016-436a-8a7c-d14b010767ba\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:21.087131 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:21.087016 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b101708b-1016-436a-8a7c-d14b010767ba-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kzr9\" (UID: \"b101708b-1016-436a-8a7c-d14b010767ba\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:21.087131 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:21.087046 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m6bk\" (UniqueName: \"kubernetes.io/projected/b101708b-1016-436a-8a7c-d14b010767ba-kube-api-access-6m6bk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kzr9\" (UID: \"b101708b-1016-436a-8a7c-d14b010767ba\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:21.087131 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:21.087062 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b101708b-1016-436a-8a7c-d14b010767ba-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kzr9\" (UID: \"b101708b-1016-436a-8a7c-d14b010767ba\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:21.087131 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:21.087113 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b101708b-1016-436a-8a7c-d14b010767ba-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kzr9\" (UID: \"b101708b-1016-436a-8a7c-d14b010767ba\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:21.087295 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:21.087136 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b101708b-1016-436a-8a7c-d14b010767ba-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kzr9\" (UID: \"b101708b-1016-436a-8a7c-d14b010767ba\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:21.087461 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:21.087439 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b101708b-1016-436a-8a7c-d14b010767ba-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kzr9\" (UID: \"b101708b-1016-436a-8a7c-d14b010767ba\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:21.087539 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:21.087452 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b101708b-1016-436a-8a7c-d14b010767ba-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kzr9\" (UID: \"b101708b-1016-436a-8a7c-d14b010767ba\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:21.087539 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:21.087495 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b101708b-1016-436a-8a7c-d14b010767ba-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kzr9\" (UID: \"b101708b-1016-436a-8a7c-d14b010767ba\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:21.089365 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:21.089346 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b101708b-1016-436a-8a7c-d14b010767ba-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kzr9\" (UID: \"b101708b-1016-436a-8a7c-d14b010767ba\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:21.089512 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:21.089497 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b101708b-1016-436a-8a7c-d14b010767ba-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kzr9\" (UID: \"b101708b-1016-436a-8a7c-d14b010767ba\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:21.099038 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:21.099014 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m6bk\" (UniqueName: \"kubernetes.io/projected/b101708b-1016-436a-8a7c-d14b010767ba-kube-api-access-6m6bk\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2kzr9\" (UID: \"b101708b-1016-436a-8a7c-d14b010767ba\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:21.242919 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:21.242844 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:21.379672 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:21.379649 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9"] Jul 06 16:54:21.382105 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:54:21.382080 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb101708b_1016_436a_8a7c_d14b010767ba.slice/crio-be7cd8763f91d7dd4b0e9e0197215bf27d9da71772c1d47a438b0e7a0997fb1d WatchSource:0}: Error finding container be7cd8763f91d7dd4b0e9e0197215bf27d9da71772c1d47a438b0e7a0997fb1d: Status 404 returned error can't find the container with id be7cd8763f91d7dd4b0e9e0197215bf27d9da71772c1d47a438b0e7a0997fb1d Jul 06 16:54:21.624779 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:21.624747 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" event={"ID":"b101708b-1016-436a-8a7c-d14b010767ba","Type":"ContainerStarted","Data":"565c76854e25517770ef40794b65c23de21d10732e0ef8951a4cf1384c062638"} Jul 06 16:54:21.624880 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:21.624801 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" event={"ID":"b101708b-1016-436a-8a7c-d14b010767ba","Type":"ContainerStarted","Data":"be7cd8763f91d7dd4b0e9e0197215bf27d9da71772c1d47a438b0e7a0997fb1d"} Jul 06 16:54:21.651662 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:21.651614 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" podStartSLOduration=1.469005636 podStartE2EDuration="1.651593555s" totalImagesPullingTime="182.587919ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:54:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:54:21.64750906 +0000 UTC m=+975.825368171" watchObservedRunningTime="2026-07-06 16:54:21.651593555 +0000 UTC m=+975.829452667" Jul 06 16:54:31.243703 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:31.243620 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:31.243703 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:31.243683 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:31.256212 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:31.256184 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:54:31.694553 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:54:31.694527 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2kzr9" Jul 06 16:56:04.678149 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:04.678114 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-566969dcc7-9dfcm"] Jul 06 16:56:04.681585 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:04.681567 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-566969dcc7-9dfcm" Jul 06 16:56:04.694065 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:04.694041 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-566969dcc7-9dfcm"] Jul 06 16:56:04.872775 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:04.872738 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7cc4fe7e-b3a2-4d2d-a800-59a904c0701a-tls-cert\") pod \"authorino-566969dcc7-9dfcm\" (UID: \"7cc4fe7e-b3a2-4d2d-a800-59a904c0701a\") " pod="kuadrant-system/authorino-566969dcc7-9dfcm" Jul 06 16:56:04.872941 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:04.872790 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4cgw\" (UniqueName: \"kubernetes.io/projected/7cc4fe7e-b3a2-4d2d-a800-59a904c0701a-kube-api-access-x4cgw\") pod \"authorino-566969dcc7-9dfcm\" (UID: \"7cc4fe7e-b3a2-4d2d-a800-59a904c0701a\") " pod="kuadrant-system/authorino-566969dcc7-9dfcm" Jul 06 16:56:04.974126 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:04.974044 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4cgw\" (UniqueName: \"kubernetes.io/projected/7cc4fe7e-b3a2-4d2d-a800-59a904c0701a-kube-api-access-x4cgw\") pod \"authorino-566969dcc7-9dfcm\" (UID: \"7cc4fe7e-b3a2-4d2d-a800-59a904c0701a\") " pod="kuadrant-system/authorino-566969dcc7-9dfcm" Jul 06 16:56:04.974270 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:04.974133 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7cc4fe7e-b3a2-4d2d-a800-59a904c0701a-tls-cert\") pod \"authorino-566969dcc7-9dfcm\" (UID: \"7cc4fe7e-b3a2-4d2d-a800-59a904c0701a\") " pod="kuadrant-system/authorino-566969dcc7-9dfcm" Jul 06 16:56:04.976545 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:04.976525 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7cc4fe7e-b3a2-4d2d-a800-59a904c0701a-tls-cert\") pod \"authorino-566969dcc7-9dfcm\" (UID: \"7cc4fe7e-b3a2-4d2d-a800-59a904c0701a\") " pod="kuadrant-system/authorino-566969dcc7-9dfcm" Jul 06 16:56:04.985759 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:04.985734 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4cgw\" (UniqueName: \"kubernetes.io/projected/7cc4fe7e-b3a2-4d2d-a800-59a904c0701a-kube-api-access-x4cgw\") pod \"authorino-566969dcc7-9dfcm\" (UID: \"7cc4fe7e-b3a2-4d2d-a800-59a904c0701a\") " pod="kuadrant-system/authorino-566969dcc7-9dfcm" Jul 06 16:56:04.990288 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:04.990268 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-566969dcc7-9dfcm" Jul 06 16:56:05.136734 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:05.136704 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-566969dcc7-9dfcm"] Jul 06 16:56:05.138592 ip-10-0-135-40 kubenswrapper[2580]: W0706 16:56:05.138558 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7cc4fe7e_b3a2_4d2d_a800_59a904c0701a.slice/crio-1b2c892a86d40fdc49926691c251d4c69835b62b38a5fefb1f1df4e0b9b432d6 WatchSource:0}: Error finding container 1b2c892a86d40fdc49926691c251d4c69835b62b38a5fefb1f1df4e0b9b432d6: Status 404 returned error can't find the container with id 1b2c892a86d40fdc49926691c251d4c69835b62b38a5fefb1f1df4e0b9b432d6 Jul 06 16:56:05.171927 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:05.171898 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-566969dcc7-9dfcm" event={"ID":"7cc4fe7e-b3a2-4d2d-a800-59a904c0701a","Type":"ContainerStarted","Data":"1b2c892a86d40fdc49926691c251d4c69835b62b38a5fefb1f1df4e0b9b432d6"} Jul 06 16:56:06.182264 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:06.182218 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-566969dcc7-9dfcm" event={"ID":"7cc4fe7e-b3a2-4d2d-a800-59a904c0701a","Type":"ContainerStarted","Data":"98872cf80569d80a823198fb3746589d416375712c80cc257f52037186e03ee4"} Jul 06 16:56:06.213915 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:06.213843 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-566969dcc7-9dfcm" podStartSLOduration=1.83413097 podStartE2EDuration="2.213825933s" totalImagesPullingTime="379.694963ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 16:56:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 16:56:06.212526043 +0000 UTC m=+1080.390385156" watchObservedRunningTime="2026-07-06 16:56:06.213825933 +0000 UTC m=+1080.391685045" Jul 06 16:56:06.258964 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:06.258926 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75b9554c88-dx6v9"] Jul 06 16:56:06.275217 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:06.275102 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75b9554c88-dx6v9" podUID="fd6d1862-61aa-41b6-a0ab-64914a1b8cd5" containerName="authorino" containerID="cri-o://6c37595469f83621fb0d2911636cd110c44fd634472db2fe2a018a2d143ff536" gracePeriod=30 Jul 06 16:56:06.546203 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:06.546172 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75b9554c88-dx6v9" Jul 06 16:56:06.592394 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:06.592363 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fd6d1862-61aa-41b6-a0ab-64914a1b8cd5-tls-cert\") pod \"fd6d1862-61aa-41b6-a0ab-64914a1b8cd5\" (UID: \"fd6d1862-61aa-41b6-a0ab-64914a1b8cd5\") " Jul 06 16:56:06.603207 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:06.603159 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd6d1862-61aa-41b6-a0ab-64914a1b8cd5-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "fd6d1862-61aa-41b6-a0ab-64914a1b8cd5" (UID: "fd6d1862-61aa-41b6-a0ab-64914a1b8cd5"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 16:56:06.692934 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:06.692897 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hnvn\" (UniqueName: \"kubernetes.io/projected/fd6d1862-61aa-41b6-a0ab-64914a1b8cd5-kube-api-access-8hnvn\") pod \"fd6d1862-61aa-41b6-a0ab-64914a1b8cd5\" (UID: \"fd6d1862-61aa-41b6-a0ab-64914a1b8cd5\") " Jul 06 16:56:06.693206 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:06.693188 2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/fd6d1862-61aa-41b6-a0ab-64914a1b8cd5-tls-cert\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:56:06.695032 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:06.695003 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd6d1862-61aa-41b6-a0ab-64914a1b8cd5-kube-api-access-8hnvn" (OuterVolumeSpecName: "kube-api-access-8hnvn") pod "fd6d1862-61aa-41b6-a0ab-64914a1b8cd5" (UID: "fd6d1862-61aa-41b6-a0ab-64914a1b8cd5"). InnerVolumeSpecName "kube-api-access-8hnvn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 16:56:06.794528 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:06.794425 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hnvn\" (UniqueName: \"kubernetes.io/projected/fd6d1862-61aa-41b6-a0ab-64914a1b8cd5-kube-api-access-8hnvn\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 16:56:07.189795 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:07.189762 2580 generic.go:356] "Generic (PLEG): container finished" podID="fd6d1862-61aa-41b6-a0ab-64914a1b8cd5" containerID="6c37595469f83621fb0d2911636cd110c44fd634472db2fe2a018a2d143ff536" exitCode=0 Jul 06 16:56:07.190227 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:07.189820 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75b9554c88-dx6v9" Jul 06 16:56:07.190227 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:07.189849 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75b9554c88-dx6v9" event={"ID":"fd6d1862-61aa-41b6-a0ab-64914a1b8cd5","Type":"ContainerDied","Data":"6c37595469f83621fb0d2911636cd110c44fd634472db2fe2a018a2d143ff536"} Jul 06 16:56:07.190227 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:07.189891 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75b9554c88-dx6v9" event={"ID":"fd6d1862-61aa-41b6-a0ab-64914a1b8cd5","Type":"ContainerDied","Data":"2018033b5426321e0faae615e02fa5d9342d8698dac35f7c46c45e24762c7825"} Jul 06 16:56:07.190227 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:07.189908 2580 scope.go:117] "RemoveContainer" containerID="6c37595469f83621fb0d2911636cd110c44fd634472db2fe2a018a2d143ff536" Jul 06 16:56:07.199194 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:07.199176 2580 scope.go:117] "RemoveContainer" containerID="6c37595469f83621fb0d2911636cd110c44fd634472db2fe2a018a2d143ff536" Jul 06 16:56:07.199449 ip-10-0-135-40 kubenswrapper[2580]: E0706 16:56:07.199430 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c37595469f83621fb0d2911636cd110c44fd634472db2fe2a018a2d143ff536\": container with ID starting with 6c37595469f83621fb0d2911636cd110c44fd634472db2fe2a018a2d143ff536 not found: ID does not exist" containerID="6c37595469f83621fb0d2911636cd110c44fd634472db2fe2a018a2d143ff536" Jul 06 16:56:07.199551 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:07.199455 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c37595469f83621fb0d2911636cd110c44fd634472db2fe2a018a2d143ff536"} err="failed to get container status \"6c37595469f83621fb0d2911636cd110c44fd634472db2fe2a018a2d143ff536\": rpc error: code = NotFound desc = could not find container \"6c37595469f83621fb0d2911636cd110c44fd634472db2fe2a018a2d143ff536\": container with ID starting with 6c37595469f83621fb0d2911636cd110c44fd634472db2fe2a018a2d143ff536 not found: ID does not exist" Jul 06 16:56:07.219601 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:07.219573 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75b9554c88-dx6v9"] Jul 06 16:56:07.221714 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:07.221688 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75b9554c88-dx6v9"] Jul 06 16:56:08.265524 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:56:08.265493 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd6d1862-61aa-41b6-a0ab-64914a1b8cd5" path="/var/lib/kubelet/pods/fd6d1862-61aa-41b6-a0ab-64914a1b8cd5/volumes" Jul 06 16:58:06.278954 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:58:06.278919 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 16:58:06.279865 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:58:06.279841 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 16:58:06.287062 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:58:06.287040 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 16:58:06.287883 ip-10-0-135-40 kubenswrapper[2580]: I0706 16:58:06.287866 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 17:03:06.316789 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:03:06.316760 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 17:03:06.319004 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:03:06.318980 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 17:03:06.321849 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:03:06.321833 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 17:03:06.323480 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:03:06.323448 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 17:08:06.350758 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:08:06.350716 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 17:08:06.354299 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:08:06.354279 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 17:08:06.355965 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:08:06.355945 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 17:08:06.359588 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:08:06.359563 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 17:13:06.388254 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:13:06.388221 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 17:13:06.394233 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:13:06.394208 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 17:13:06.396090 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:13:06.396072 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 17:13:06.401257 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:13:06.401231 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 17:18:06.426033 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:18:06.425997 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 17:18:06.431336 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:18:06.431311 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 17:18:06.442731 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:18:06.442711 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 17:18:06.447648 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:18:06.447632 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 17:23:06.477821 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:06.477736 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 17:23:06.483550 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:06.483532 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 17:23:06.484211 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:06.484190 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 17:23:06.489871 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:06.489854 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jrb4n_478a85da-6e22-4a45-8ab8-bd99215ec846/ovn-acl-logging/0.log" Jul 06 17:23:54.805999 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:54.805966 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk"] Jul 06 17:23:54.806450 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:54.806313 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd6d1862-61aa-41b6-a0ab-64914a1b8cd5" containerName="authorino" Jul 06 17:23:54.806450 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:54.806325 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd6d1862-61aa-41b6-a0ab-64914a1b8cd5" containerName="authorino" Jul 06 17:23:54.806450 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:54.806388 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd6d1862-61aa-41b6-a0ab-64914a1b8cd5" containerName="authorino" Jul 06 17:23:54.809399 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:54.809383 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:23:54.813806 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:54.813774 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-2cd1eb\"/\"default-dockercfg-slxr4\"" Jul 06 17:23:54.813930 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:54.813836 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-2cd1eb\"/\"kube-root-ca.crt\"" Jul 06 17:23:54.813930 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:54.813774 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-2cd1eb\"/\"rate-test-model-2cd1eb-kserve-self-signed-certs\"" Jul 06 17:23:54.813930 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:54.813887 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-2cd1eb\"/\"openshift-service-ca.crt\"" Jul 06 17:23:54.822038 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:54.822014 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk"] Jul 06 17:23:54.949363 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:54.949330 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-model-cache\") pod \"rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:23:54.949363 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:54.949361 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc4d2518-ff90-4844-9738-ec08d27b263f-tls-certs\") pod \"rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:23:54.949610 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:54.949399 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-home\") pod \"rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:23:54.949610 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:54.949496 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-dshm\") pod \"rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:23:54.949610 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:54.949565 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7cfj\" (UniqueName: \"kubernetes.io/projected/fc4d2518-ff90-4844-9738-ec08d27b263f-kube-api-access-s7cfj\") pod \"rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:23:54.949610 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:54.949600 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-tmp-dir\") pod \"rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:23:55.050689 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:55.050658 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7cfj\" (UniqueName: \"kubernetes.io/projected/fc4d2518-ff90-4844-9738-ec08d27b263f-kube-api-access-s7cfj\") pod \"rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:23:55.050689 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:55.050693 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-tmp-dir\") pod \"rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:23:55.050931 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:55.050733 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-model-cache\") pod \"rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:23:55.050931 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:55.050749 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc4d2518-ff90-4844-9738-ec08d27b263f-tls-certs\") pod \"rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:23:55.050931 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:55.050791 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-home\") pod \"rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:23:55.050931 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:55.050816 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-dshm\") pod \"rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:23:55.051169 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:55.051116 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-tmp-dir\") pod \"rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:23:55.051214 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:55.051174 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-model-cache\") pod \"rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:23:55.051214 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:55.051193 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-home\") pod \"rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:23:55.052969 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:55.052940 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-dshm\") pod \"rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:23:55.053200 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:55.053183 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc4d2518-ff90-4844-9738-ec08d27b263f-tls-certs\") pod \"rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:23:55.058822 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:55.058767 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7cfj\" (UniqueName: \"kubernetes.io/projected/fc4d2518-ff90-4844-9738-ec08d27b263f-kube-api-access-s7cfj\") pod \"rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:23:55.118780 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:55.118752 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:23:55.251991 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:55.251955 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk"] Jul 06 17:23:55.253686 ip-10-0-135-40 kubenswrapper[2580]: W0706 17:23:55.253661 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc4d2518_ff90_4844_9738_ec08d27b263f.slice/crio-67df5c694fdbeecc1973ee533a8938a6ee8cd17ded047365d7139a6b20952b56 WatchSource:0}: Error finding container 67df5c694fdbeecc1973ee533a8938a6ee8cd17ded047365d7139a6b20952b56: Status 404 returned error can't find the container with id 67df5c694fdbeecc1973ee533a8938a6ee8cd17ded047365d7139a6b20952b56 Jul 06 17:23:55.923414 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:55.923365 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" event={"ID":"fc4d2518-ff90-4844-9738-ec08d27b263f","Type":"ContainerStarted","Data":"1d7c628c0c7207cef66782845fcf9eae01d4c565da2f170584026c6a827e547d"} Jul 06 17:23:55.923414 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:55.923416 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" event={"ID":"fc4d2518-ff90-4844-9738-ec08d27b263f","Type":"ContainerStarted","Data":"67df5c694fdbeecc1973ee533a8938a6ee8cd17ded047365d7139a6b20952b56"} Jul 06 17:23:55.942595 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:23:55.942555 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" podStartSLOduration=1.9425415 podStartE2EDuration="1.9425415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:23:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:23:55.940128354 +0000 UTC m=+2750.117987470" watchObservedRunningTime="2026-07-06 17:23:55.9425415 +0000 UTC m=+2750.120400612" Jul 06 17:24:05.119017 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:24:05.118978 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:24:05.119017 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:24:05.119027 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:24:05.131740 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:24:05.131715 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:24:05.987488 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:24:05.987439 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:25:00.948775 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:00.948739 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk"] Jul 06 17:25:15.968939 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:15.968900 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" podUID="fc4d2518-ff90-4844-9738-ec08d27b263f" containerName="main" containerID="cri-o://1d7c628c0c7207cef66782845fcf9eae01d4c565da2f170584026c6a827e547d" gracePeriod=45 Jul 06 17:25:15.977181 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:15.977142 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" podUID="fc4d2518-ff90-4844-9738-ec08d27b263f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.46:8000/ready\": dial tcp 10.132.0.46:8000: connect: connection refused" Jul 06 17:25:16.220982 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.220929 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:25:16.343504 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.343456 2580 generic.go:356] "Generic (PLEG): container finished" podID="fc4d2518-ff90-4844-9738-ec08d27b263f" containerID="1d7c628c0c7207cef66782845fcf9eae01d4c565da2f170584026c6a827e547d" exitCode=0 Jul 06 17:25:16.343753 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.343530 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" Jul 06 17:25:16.343753 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.343534 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" event={"ID":"fc4d2518-ff90-4844-9738-ec08d27b263f","Type":"ContainerDied","Data":"1d7c628c0c7207cef66782845fcf9eae01d4c565da2f170584026c6a827e547d"} Jul 06 17:25:16.343753 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.343575 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk" event={"ID":"fc4d2518-ff90-4844-9738-ec08d27b263f","Type":"ContainerDied","Data":"67df5c694fdbeecc1973ee533a8938a6ee8cd17ded047365d7139a6b20952b56"} Jul 06 17:25:16.343753 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.343594 2580 scope.go:117] "RemoveContainer" containerID="1d7c628c0c7207cef66782845fcf9eae01d4c565da2f170584026c6a827e547d" Jul 06 17:25:16.352665 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.352644 2580 scope.go:117] "RemoveContainer" containerID="1d7c628c0c7207cef66782845fcf9eae01d4c565da2f170584026c6a827e547d" Jul 06 17:25:16.352968 ip-10-0-135-40 kubenswrapper[2580]: E0706 17:25:16.352941 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d7c628c0c7207cef66782845fcf9eae01d4c565da2f170584026c6a827e547d\": container with ID starting with 1d7c628c0c7207cef66782845fcf9eae01d4c565da2f170584026c6a827e547d not found: ID does not exist" containerID="1d7c628c0c7207cef66782845fcf9eae01d4c565da2f170584026c6a827e547d" Jul 06 17:25:16.353063 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.352999 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d7c628c0c7207cef66782845fcf9eae01d4c565da2f170584026c6a827e547d"} err="failed to get container status \"1d7c628c0c7207cef66782845fcf9eae01d4c565da2f170584026c6a827e547d\": rpc error: code = NotFound desc = could not find container \"1d7c628c0c7207cef66782845fcf9eae01d4c565da2f170584026c6a827e547d\": container with ID starting with 1d7c628c0c7207cef66782845fcf9eae01d4c565da2f170584026c6a827e547d not found: ID does not exist" Jul 06 17:25:16.388353 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.388327 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-dshm\") pod \"fc4d2518-ff90-4844-9738-ec08d27b263f\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " Jul 06 17:25:16.388499 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.388366 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-tmp-dir\") pod \"fc4d2518-ff90-4844-9738-ec08d27b263f\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " Jul 06 17:25:16.388499 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.388411 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-home\") pod \"fc4d2518-ff90-4844-9738-ec08d27b263f\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " Jul 06 17:25:16.388499 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.388438 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7cfj\" (UniqueName: \"kubernetes.io/projected/fc4d2518-ff90-4844-9738-ec08d27b263f-kube-api-access-s7cfj\") pod \"fc4d2518-ff90-4844-9738-ec08d27b263f\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " Jul 06 17:25:16.388499 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.388460 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc4d2518-ff90-4844-9738-ec08d27b263f-tls-certs\") pod \"fc4d2518-ff90-4844-9738-ec08d27b263f\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " Jul 06 17:25:16.388728 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.388559 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-model-cache\") pod \"fc4d2518-ff90-4844-9738-ec08d27b263f\" (UID: \"fc4d2518-ff90-4844-9738-ec08d27b263f\") " Jul 06 17:25:16.388728 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.388704 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fc4d2518-ff90-4844-9738-ec08d27b263f" (UID: "fc4d2518-ff90-4844-9738-ec08d27b263f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 17:25:16.388837 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.388720 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-home" (OuterVolumeSpecName: "home") pod "fc4d2518-ff90-4844-9738-ec08d27b263f" (UID: "fc4d2518-ff90-4844-9738-ec08d27b263f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 17:25:16.388889 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.388850 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-model-cache" (OuterVolumeSpecName: "model-cache") pod "fc4d2518-ff90-4844-9738-ec08d27b263f" (UID: "fc4d2518-ff90-4844-9738-ec08d27b263f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 17:25:16.388956 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.388937 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-tmp-dir\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 17:25:16.389010 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.388962 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-home\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 17:25:16.389010 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.388976 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-model-cache\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 17:25:16.390527 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.390500 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-dshm" (OuterVolumeSpecName: "dshm") pod "fc4d2518-ff90-4844-9738-ec08d27b263f" (UID: "fc4d2518-ff90-4844-9738-ec08d27b263f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 17:25:16.390684 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.390658 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc4d2518-ff90-4844-9738-ec08d27b263f-kube-api-access-s7cfj" (OuterVolumeSpecName: "kube-api-access-s7cfj") pod "fc4d2518-ff90-4844-9738-ec08d27b263f" (UID: "fc4d2518-ff90-4844-9738-ec08d27b263f"). InnerVolumeSpecName "kube-api-access-s7cfj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 17:25:16.390743 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.390679 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc4d2518-ff90-4844-9738-ec08d27b263f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fc4d2518-ff90-4844-9738-ec08d27b263f" (UID: "fc4d2518-ff90-4844-9738-ec08d27b263f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 17:25:16.490432 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.490353 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7cfj\" (UniqueName: \"kubernetes.io/projected/fc4d2518-ff90-4844-9738-ec08d27b263f-kube-api-access-s7cfj\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 17:25:16.490432 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.490379 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc4d2518-ff90-4844-9738-ec08d27b263f-tls-certs\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 17:25:16.490432 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.490391 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc4d2518-ff90-4844-9738-ec08d27b263f-dshm\") on node \"ip-10-0-135-40.ec2.internal\" DevicePath \"\"" Jul 06 17:25:16.678149 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.678120 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk"] Jul 06 17:25:16.691948 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:16.691920 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-2cd1eb/rate-test-model-2cd1eb-kserve-75c8d455bc-bg4lk"] Jul 06 17:25:18.264103 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:18.264066 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc4d2518-ff90-4844-9738-ec08d27b263f" path="/var/lib/kubelet/pods/fc4d2518-ff90-4844-9738-ec08d27b263f/volumes" Jul 06 17:25:54.894459 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:25:54.894427 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-566969dcc7-9dfcm_7cc4fe7e-b3a2-4d2d-a800-59a904c0701a/authorino/0.log" Jul 06 17:26:00.254006 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:00.253930 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-blccs_e1f5e726-3704-461f-a075-7a2a791f1dfc/server/0.log" Jul 06 17:26:00.371416 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:00.371383 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-576h9_fc68b1a1-028e-4d77-a084-c17a8757bc79/manager/0.log" Jul 06 17:26:00.483581 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:00.483552 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d66bcd77d-4pr8j_245f6ba2-b372-4443-895c-4237ac62053a/copy-manifests/0.log" Jul 06 17:26:00.501408 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:00.501381 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d66bcd77d-4pr8j_245f6ba2-b372-4443-895c-4237ac62053a/manager/0.log" Jul 06 17:26:01.602525 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:01.602497 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc_abcfeaa2-8dc0-45bc-a4b3-d96654b78504/util/0.log" Jul 06 17:26:01.608839 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:01.608813 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc_abcfeaa2-8dc0-45bc-a4b3-d96654b78504/pull/0.log" Jul 06 17:26:01.614300 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:01.614279 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc_abcfeaa2-8dc0-45bc-a4b3-d96654b78504/extract/0.log" Jul 06 17:26:01.729972 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:01.729943 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx_dcfa17b3-63be-4f41-925b-adf8dd903fa7/util/0.log" Jul 06 17:26:01.738293 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:01.738263 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx_dcfa17b3-63be-4f41-925b-adf8dd903fa7/pull/0.log" Jul 06 17:26:01.745786 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:01.745760 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx_dcfa17b3-63be-4f41-925b-adf8dd903fa7/extract/0.log" Jul 06 17:26:01.861387 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:01.861306 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp_743eeb1b-83ee-435d-886e-bed15f92ea77/util/0.log" Jul 06 17:26:01.869292 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:01.869268 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp_743eeb1b-83ee-435d-886e-bed15f92ea77/pull/0.log" Jul 06 17:26:01.878900 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:01.878872 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp_743eeb1b-83ee-435d-886e-bed15f92ea77/extract/0.log" Jul 06 17:26:01.985236 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:01.985209 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw_bb983e83-732f-4b99-bfa8-8b774db1f6d5/util/0.log" Jul 06 17:26:01.991956 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:01.991930 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw_bb983e83-732f-4b99-bfa8-8b774db1f6d5/pull/0.log" Jul 06 17:26:01.997855 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:01.997836 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw_bb983e83-732f-4b99-bfa8-8b774db1f6d5/extract/0.log" Jul 06 17:26:02.121495 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:02.121397 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-566969dcc7-9dfcm_7cc4fe7e-b3a2-4d2d-a800-59a904c0701a/authorino/0.log" Jul 06 17:26:03.271663 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:03.271629 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678gfmz6_3a834f5d-a321-49e6-afe4-5d054c1ee3fc/istio-proxy/0.log" Jul 06 17:26:03.613038 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:03.613003 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5548f7d4db-lcsdz_152039b3-78fb-464d-8915-50995f2c6b35/kube-auth-proxy/0.log" Jul 06 17:26:04.111521 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:04.111493 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-87b54f4c7-4lpwt_f7221e3e-0a2d-4d91-b0f1-1b303255b302/router/0.log" Jul 06 17:26:04.442460 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:04.442376 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-jtzns_a8d2b9b9-75ac-4297-9207-de2cd1dc876d/main/0.log" Jul 06 17:26:04.906941 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:04.906910 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-2kzr9_b101708b-1016-436a-8a7c-d14b010767ba/main/0.log" Jul 06 17:26:09.118887 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:09.118845 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hztcg/must-gather-85v9x"] Jul 06 17:26:09.119403 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:09.119382 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc4d2518-ff90-4844-9738-ec08d27b263f" containerName="main" Jul 06 17:26:09.119516 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:09.119405 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc4d2518-ff90-4844-9738-ec08d27b263f" containerName="main" Jul 06 17:26:09.119584 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:09.119531 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc4d2518-ff90-4844-9738-ec08d27b263f" containerName="main" Jul 06 17:26:09.124153 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:09.124122 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hztcg/must-gather-85v9x" Jul 06 17:26:09.126236 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:09.126213 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hztcg\"/\"default-dockercfg-7dvzm\"" Jul 06 17:26:09.126715 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:09.126697 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hztcg\"/\"kube-root-ca.crt\"" Jul 06 17:26:09.126791 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:09.126724 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hztcg\"/\"openshift-service-ca.crt\"" Jul 06 17:26:09.130752 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:09.130731 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hztcg/must-gather-85v9x"] Jul 06 17:26:09.234291 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:09.234254 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3a1da4ed-6f6e-4b5a-b6fc-0596662ed4e4-must-gather-output\") pod \"must-gather-85v9x\" (UID: \"3a1da4ed-6f6e-4b5a-b6fc-0596662ed4e4\") " pod="openshift-must-gather-hztcg/must-gather-85v9x" Jul 06 17:26:09.234461 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:09.234341 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nppj\" (UniqueName: \"kubernetes.io/projected/3a1da4ed-6f6e-4b5a-b6fc-0596662ed4e4-kube-api-access-6nppj\") pod \"must-gather-85v9x\" (UID: \"3a1da4ed-6f6e-4b5a-b6fc-0596662ed4e4\") " pod="openshift-must-gather-hztcg/must-gather-85v9x" Jul 06 17:26:09.334861 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:09.334822 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3a1da4ed-6f6e-4b5a-b6fc-0596662ed4e4-must-gather-output\") pod \"must-gather-85v9x\" (UID: \"3a1da4ed-6f6e-4b5a-b6fc-0596662ed4e4\") " pod="openshift-must-gather-hztcg/must-gather-85v9x" Jul 06 17:26:09.334861 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:09.334868 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nppj\" (UniqueName: \"kubernetes.io/projected/3a1da4ed-6f6e-4b5a-b6fc-0596662ed4e4-kube-api-access-6nppj\") pod \"must-gather-85v9x\" (UID: \"3a1da4ed-6f6e-4b5a-b6fc-0596662ed4e4\") " pod="openshift-must-gather-hztcg/must-gather-85v9x" Jul 06 17:26:09.335206 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:09.335186 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/3a1da4ed-6f6e-4b5a-b6fc-0596662ed4e4-must-gather-output\") pod \"must-gather-85v9x\" (UID: \"3a1da4ed-6f6e-4b5a-b6fc-0596662ed4e4\") " pod="openshift-must-gather-hztcg/must-gather-85v9x" Jul 06 17:26:09.347074 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:09.347042 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nppj\" (UniqueName: \"kubernetes.io/projected/3a1da4ed-6f6e-4b5a-b6fc-0596662ed4e4-kube-api-access-6nppj\") pod \"must-gather-85v9x\" (UID: \"3a1da4ed-6f6e-4b5a-b6fc-0596662ed4e4\") " pod="openshift-must-gather-hztcg/must-gather-85v9x" Jul 06 17:26:09.434636 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:09.434564 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hztcg/must-gather-85v9x" Jul 06 17:26:09.568459 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:09.568431 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hztcg/must-gather-85v9x"] Jul 06 17:26:09.569747 ip-10-0-135-40 kubenswrapper[2580]: W0706 17:26:09.569719 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a1da4ed_6f6e_4b5a_b6fc_0596662ed4e4.slice/crio-2fd13dba52437b2a3a4b9808b827669b5ae2b95e2ad11ea70e235719fab1e8bb WatchSource:0}: Error finding container 2fd13dba52437b2a3a4b9808b827669b5ae2b95e2ad11ea70e235719fab1e8bb: Status 404 returned error can't find the container with id 2fd13dba52437b2a3a4b9808b827669b5ae2b95e2ad11ea70e235719fab1e8bb Jul 06 17:26:09.571444 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:09.571428 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 17:26:09.623476 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:09.623445 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hztcg/must-gather-85v9x" event={"ID":"3a1da4ed-6f6e-4b5a-b6fc-0596662ed4e4","Type":"ContainerStarted","Data":"2fd13dba52437b2a3a4b9808b827669b5ae2b95e2ad11ea70e235719fab1e8bb"} Jul 06 17:26:11.637996 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:11.637959 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hztcg/must-gather-85v9x" event={"ID":"3a1da4ed-6f6e-4b5a-b6fc-0596662ed4e4","Type":"ContainerStarted","Data":"8362e0b9b879f8a327d18eec44f97c11af3710f8a48131ce4cf7406d7f37e3c9"} Jul 06 17:26:11.638381 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:11.638006 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hztcg/must-gather-85v9x" event={"ID":"3a1da4ed-6f6e-4b5a-b6fc-0596662ed4e4","Type":"ContainerStarted","Data":"94c448b695c7e595835582e8f7a6c976cd9038962ac656d13e0e844f8e71d57e"} Jul 06 17:26:11.656246 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:11.656201 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hztcg/must-gather-85v9x" podStartSLOduration=1.491401483 podStartE2EDuration="2.656187155s" totalImagesPullingTime="1.164785672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 17:26:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:26:11.655311508 +0000 UTC m=+2885.833170621" watchObservedRunningTime="2026-07-06 17:26:11.656187155 +0000 UTC m=+2885.834046300" Jul 06 17:26:12.403693 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:12.403658 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2m87r_0f5cbd3c-3a54-4441-bea2-2643a5912979/global-pull-secret-syncer/0.log" Jul 06 17:26:12.611434 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:12.611400 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xqf97_66c85ad6-1024-423c-b7bc-2c17115326c1/konnectivity-agent/0.log" Jul 06 17:26:12.737236 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:12.737145 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-40.ec2.internal_b5e29f9f4d849ce3821c6ee834e1493a/haproxy/0.log" Jul 06 17:26:17.175283 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:17.175252 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc_abcfeaa2-8dc0-45bc-a4b3-d96654b78504/extract/0.log" Jul 06 17:26:17.227305 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:17.227275 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc_abcfeaa2-8dc0-45bc-a4b3-d96654b78504/util/0.log" Jul 06 17:26:17.279079 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:17.279030 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598czsc_abcfeaa2-8dc0-45bc-a4b3-d96654b78504/pull/0.log" Jul 06 17:26:17.333462 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:17.333426 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx_dcfa17b3-63be-4f41-925b-adf8dd903fa7/extract/0.log" Jul 06 17:26:17.375036 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:17.375007 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx_dcfa17b3-63be-4f41-925b-adf8dd903fa7/util/0.log" Jul 06 17:26:17.409590 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:17.409563 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e08phjx_dcfa17b3-63be-4f41-925b-adf8dd903fa7/pull/0.log" Jul 06 17:26:17.465099 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:17.465014 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp_743eeb1b-83ee-435d-886e-bed15f92ea77/extract/0.log" Jul 06 17:26:17.498940 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:17.498909 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp_743eeb1b-83ee-435d-886e-bed15f92ea77/util/0.log" Jul 06 17:26:17.540660 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:17.540621 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73hghbp_743eeb1b-83ee-435d-886e-bed15f92ea77/pull/0.log" Jul 06 17:26:17.585515 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:17.585456 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw_bb983e83-732f-4b99-bfa8-8b774db1f6d5/extract/0.log" Jul 06 17:26:17.622935 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:17.622829 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw_bb983e83-732f-4b99-bfa8-8b774db1f6d5/util/0.log" Jul 06 17:26:17.662374 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:17.662332 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef178tmw_bb983e83-732f-4b99-bfa8-8b774db1f6d5/pull/0.log" Jul 06 17:26:18.151952 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:18.151920 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-566969dcc7-9dfcm_7cc4fe7e-b3a2-4d2d-a800-59a904c0701a/authorino/0.log" Jul 06 17:26:20.549893 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:20.549854 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-x247s_59df778e-ece6-45e2-a0df-551df597ee79/cluster-monitoring-operator/0.log" Jul 06 17:26:20.682884 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:20.682850 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-cnv9g_2fb77d8e-423d-41f9-90ec-a9f18b514ea5/monitoring-plugin/0.log" Jul 06 17:26:20.877101 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:20.877067 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v4bch_173ca302-b638-42e3-891e-467c6a727dd9/node-exporter/0.log" Jul 06 17:26:20.904079 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:20.904047 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v4bch_173ca302-b638-42e3-891e-467c6a727dd9/kube-rbac-proxy/0.log" Jul 06 17:26:20.926300 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:20.926276 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v4bch_173ca302-b638-42e3-891e-467c6a727dd9/init-textfile/0.log" Jul 06 17:26:21.381739 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:21.381700 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-64wml_0be6285e-fbac-4d35-8888-431e53289a2f/prometheus-operator/0.log" Jul 06 17:26:21.430286 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:21.430238 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-64wml_0be6285e-fbac-4d35-8888-431e53289a2f/kube-rbac-proxy/0.log" Jul 06 17:26:21.471708 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:21.471673 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-42c8w_eb8b2f4a-3d04-4aa1-beb9-02be75dc54c7/prometheus-operator-admission-webhook/0.log" Jul 06 17:26:21.637376 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:21.637283 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-747b755857-hhwcw_e136234c-51b1-45cf-8127-f4585f548774/thanos-query/0.log" Jul 06 17:26:21.665051 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:21.665017 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-747b755857-hhwcw_e136234c-51b1-45cf-8127-f4585f548774/kube-rbac-proxy-web/0.log" Jul 06 17:26:21.698016 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:21.697983 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-747b755857-hhwcw_e136234c-51b1-45cf-8127-f4585f548774/kube-rbac-proxy/0.log" Jul 06 17:26:21.733530 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:21.733500 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-747b755857-hhwcw_e136234c-51b1-45cf-8127-f4585f548774/prom-label-proxy/0.log" Jul 06 17:26:21.775530 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:21.775505 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-747b755857-hhwcw_e136234c-51b1-45cf-8127-f4585f548774/kube-rbac-proxy-rules/0.log" Jul 06 17:26:21.819902 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:21.819872 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-747b755857-hhwcw_e136234c-51b1-45cf-8127-f4585f548774/kube-rbac-proxy-metrics/0.log" Jul 06 17:26:22.269838 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:22.269809 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn"] Jul 06 17:26:22.276190 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:22.276163 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" Jul 06 17:26:22.283036 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:22.283008 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn"] Jul 06 17:26:22.376927 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:22.376887 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e20cc6ee-f374-402d-8d77-cbeb759368ec-sys\") pod \"perf-node-gather-daemonset-5hrfn\" (UID: \"e20cc6ee-f374-402d-8d77-cbeb759368ec\") " pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" Jul 06 17:26:22.377125 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:22.377036 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjx9h\" (UniqueName: \"kubernetes.io/projected/e20cc6ee-f374-402d-8d77-cbeb759368ec-kube-api-access-pjx9h\") pod \"perf-node-gather-daemonset-5hrfn\" (UID: \"e20cc6ee-f374-402d-8d77-cbeb759368ec\") " pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" Jul 06 17:26:22.377125 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:22.377077 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e20cc6ee-f374-402d-8d77-cbeb759368ec-proc\") pod \"perf-node-gather-daemonset-5hrfn\" (UID: \"e20cc6ee-f374-402d-8d77-cbeb759368ec\") " pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" Jul 06 17:26:22.377255 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:22.377152 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e20cc6ee-f374-402d-8d77-cbeb759368ec-lib-modules\") pod \"perf-node-gather-daemonset-5hrfn\" (UID: \"e20cc6ee-f374-402d-8d77-cbeb759368ec\") " pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" Jul 06 17:26:22.377255 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:22.377171 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e20cc6ee-f374-402d-8d77-cbeb759368ec-podres\") pod \"perf-node-gather-daemonset-5hrfn\" (UID: \"e20cc6ee-f374-402d-8d77-cbeb759368ec\") " pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" Jul 06 17:26:22.478327 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:22.478286 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e20cc6ee-f374-402d-8d77-cbeb759368ec-lib-modules\") pod \"perf-node-gather-daemonset-5hrfn\" (UID: \"e20cc6ee-f374-402d-8d77-cbeb759368ec\") " pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" Jul 06 17:26:22.478327 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:22.478326 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e20cc6ee-f374-402d-8d77-cbeb759368ec-podres\") pod \"perf-node-gather-daemonset-5hrfn\" (UID: \"e20cc6ee-f374-402d-8d77-cbeb759368ec\") " pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" Jul 06 17:26:22.478610 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:22.478357 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e20cc6ee-f374-402d-8d77-cbeb759368ec-sys\") pod \"perf-node-gather-daemonset-5hrfn\" (UID: \"e20cc6ee-f374-402d-8d77-cbeb759368ec\") " pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" Jul 06 17:26:22.478610 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:22.478403 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjx9h\" (UniqueName: \"kubernetes.io/projected/e20cc6ee-f374-402d-8d77-cbeb759368ec-kube-api-access-pjx9h\") pod \"perf-node-gather-daemonset-5hrfn\" (UID: \"e20cc6ee-f374-402d-8d77-cbeb759368ec\") " pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" Jul 06 17:26:22.478610 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:22.478427 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e20cc6ee-f374-402d-8d77-cbeb759368ec-proc\") pod \"perf-node-gather-daemonset-5hrfn\" (UID: \"e20cc6ee-f374-402d-8d77-cbeb759368ec\") " pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" Jul 06 17:26:22.478610 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:22.478508 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e20cc6ee-f374-402d-8d77-cbeb759368ec-podres\") pod \"perf-node-gather-daemonset-5hrfn\" (UID: \"e20cc6ee-f374-402d-8d77-cbeb759368ec\") " pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" Jul 06 17:26:22.478610 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:22.478537 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e20cc6ee-f374-402d-8d77-cbeb759368ec-proc\") pod \"perf-node-gather-daemonset-5hrfn\" (UID: \"e20cc6ee-f374-402d-8d77-cbeb759368ec\") " pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" Jul 06 17:26:22.478610 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:22.478508 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e20cc6ee-f374-402d-8d77-cbeb759368ec-lib-modules\") pod \"perf-node-gather-daemonset-5hrfn\" (UID: \"e20cc6ee-f374-402d-8d77-cbeb759368ec\") " pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" Jul 06 17:26:22.478610 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:22.478551 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e20cc6ee-f374-402d-8d77-cbeb759368ec-sys\") pod \"perf-node-gather-daemonset-5hrfn\" (UID: \"e20cc6ee-f374-402d-8d77-cbeb759368ec\") " pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" Jul 06 17:26:22.489648 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:22.489622 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjx9h\" (UniqueName: \"kubernetes.io/projected/e20cc6ee-f374-402d-8d77-cbeb759368ec-kube-api-access-pjx9h\") pod \"perf-node-gather-daemonset-5hrfn\" (UID: \"e20cc6ee-f374-402d-8d77-cbeb759368ec\") " pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" Jul 06 17:26:22.589844 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:22.589797 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" Jul 06 17:26:22.774918 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:22.774600 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn"] Jul 06 17:26:22.778306 ip-10-0-135-40 kubenswrapper[2580]: W0706 17:26:22.778275 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode20cc6ee_f374_402d_8d77_cbeb759368ec.slice/crio-a485ab99fd64b1167289c9d474da62ca323dec79c288f99d72f1c658ececb130 WatchSource:0}: Error finding container a485ab99fd64b1167289c9d474da62ca323dec79c288f99d72f1c658ececb130: Status 404 returned error can't find the container with id a485ab99fd64b1167289c9d474da62ca323dec79c288f99d72f1c658ececb130 Jul 06 17:26:23.745392 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:23.745349 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" event={"ID":"e20cc6ee-f374-402d-8d77-cbeb759368ec","Type":"ContainerStarted","Data":"bb294f575ad33464fd75da775b57ff36dbfc720f71aaa2eacd09bec8b1ece362"} Jul 06 17:26:23.745582 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:23.745421 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" event={"ID":"e20cc6ee-f374-402d-8d77-cbeb759368ec","Type":"ContainerStarted","Data":"a485ab99fd64b1167289c9d474da62ca323dec79c288f99d72f1c658ececb130"} Jul 06 17:26:23.746234 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:23.746209 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" Jul 06 17:26:23.799801 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:23.799745 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" podStartSLOduration=1.79973033 podStartE2EDuration="1.79973033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 17:26:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 17:26:23.79799668 +0000 UTC m=+2897.975855791" watchObservedRunningTime="2026-07-06 17:26:23.79973033 +0000 UTC m=+2897.977589441" Jul 06 17:26:23.859097 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:23.859062 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/1.log" Jul 06 17:26:23.873045 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:23.873012 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-pth6b_44d046d2-51c5-4769-a665-1805b2c28340/console-operator/2.log" Jul 06 17:26:25.301366 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:25.301275 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-mct68_5d998ace-6b9c-435e-9360-b0776df34e66/volume-data-source-validator/0.log" Jul 06 17:26:26.533720 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:26.533690 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mrznn_d0604265-3597-4501-9b25-8765c41d7f49/dns/0.log" Jul 06 17:26:26.565588 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:26.565560 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-mrznn_d0604265-3597-4501-9b25-8765c41d7f49/kube-rbac-proxy/0.log" Jul 06 17:26:26.849638 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:26.849600 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-q4flx_cf835c0c-f2ea-416f-a27f-619d7eeaf9c6/dns-node-resolver/0.log" Jul 06 17:26:27.675701 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:27.675671 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zw578_206b0067-d25e-40fd-ab2d-5a834df0c969/node-ca/0.log" Jul 06 17:26:28.719850 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:28.719820 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678gfmz6_3a834f5d-a321-49e6-afe4-5d054c1ee3fc/istio-proxy/0.log" Jul 06 17:26:28.878494 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:28.878449 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5548f7d4db-lcsdz_152039b3-78fb-464d-8915-50995f2c6b35/kube-auth-proxy/0.log" Jul 06 17:26:29.096213 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:29.096183 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-87b54f4c7-4lpwt_f7221e3e-0a2d-4d91-b0f1-1b303255b302/router/0.log" Jul 06 17:26:29.834792 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:29.834758 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4xll7_f13c431e-6bae-4714-a1ba-49fa1c767244/serve-healthcheck-canary/0.log" Jul 06 17:26:30.374278 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:30.374243 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-cdh55_b6aabdd3-9fd3-46e0-ac1b-9636787ae735/insights-operator/0.log" Jul 06 17:26:30.377169 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:30.377148 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-cdh55_b6aabdd3-9fd3-46e0-ac1b-9636787ae735/insights-operator/1.log" Jul 06 17:26:30.587006 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:30.586976 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ps5hn_6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e/kube-rbac-proxy/0.log" Jul 06 17:26:30.609843 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:30.609815 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ps5hn_6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e/exporter/0.log" Jul 06 17:26:30.633136 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:30.633065 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-ps5hn_6046edef-fd1a-4e09-a7f5-a6b9a3f70c8e/extractor/0.log" Jul 06 17:26:30.770426 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:30.770389 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hztcg/perf-node-gather-daemonset-5hrfn" Jul 06 17:26:33.054011 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:33.053981 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678gfmz6_3a834f5d-a321-49e6-afe4-5d054c1ee3fc/istio-proxy/0.log" Jul 06 17:26:33.377554 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:33.377530 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5548f7d4db-lcsdz_152039b3-78fb-464d-8915-50995f2c6b35/kube-auth-proxy/0.log" Jul 06 17:26:33.715160 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:33.715133 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-87b54f4c7-4lpwt_f7221e3e-0a2d-4d91-b0f1-1b303255b302/router/0.log" Jul 06 17:26:33.878909 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:33.878885 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-blccs_e1f5e726-3704-461f-a075-7a2a791f1dfc/server/0.log" Jul 06 17:26:33.995923 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:33.995886 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-576h9_fc68b1a1-028e-4d77-a084-c17a8757bc79/manager/0.log" Jul 06 17:26:34.153725 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:34.153639 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d66bcd77d-4pr8j_245f6ba2-b372-4443-895c-4237ac62053a/manager/0.log" Jul 06 17:26:34.189086 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:34.189055 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d66bcd77d-4pr8j_245f6ba2-b372-4443-895c-4237ac62053a/copy-manifests/0.log" Jul 06 17:26:34.957301 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:34.957265 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678gfmz6_3a834f5d-a321-49e6-afe4-5d054c1ee3fc/istio-proxy/0.log" Jul 06 17:26:35.101026 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:35.101000 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5548f7d4db-lcsdz_152039b3-78fb-464d-8915-50995f2c6b35/kube-auth-proxy/0.log" Jul 06 17:26:35.461518 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:35.461484 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-87b54f4c7-4lpwt_f7221e3e-0a2d-4d91-b0f1-1b303255b302/router/0.log" Jul 06 17:26:36.123527 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:36.123494 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-5c47b9fd86-qfqfm_6a6976fd-d3e3-47c7-b419-55facb3761a5/manager/0.log" Jul 06 17:26:36.266547 ip-10-0-135-40 kubenswrapper[2580]: I0706 17:26:36.266511 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-lnwhq_bbdb8a1e-75d3-40ba-94a9-36f63fe4788c/openshift-lws-operator/0.log"