Jul 28 17:10:15.165733 ip-10-0-134-14 systemd[1]: Starting Kubernetes Kubelet... Jul 28 17:10:15.512591 ip-10-0-134-14 kubenswrapper[2550]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 17:10:15.512591 ip-10-0-134-14 kubenswrapper[2550]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 28 17:10:15.512591 ip-10-0-134-14 kubenswrapper[2550]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 17:10:15.512591 ip-10-0-134-14 kubenswrapper[2550]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 28 17:10:15.512591 ip-10-0-134-14 kubenswrapper[2550]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 28 17:10:15.514733 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.514637 2550 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 28 17:10:15.518483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518466 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:10:15.518483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518483 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:10:15.518549 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518488 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:10:15.518549 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518494 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:10:15.518549 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518503 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:10:15.518549 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518507 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:10:15.518549 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518510 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:10:15.518549 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518513 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:10:15.518549 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518515 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:10:15.518549 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518518 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:10:15.518549 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518520 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:10:15.518549 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518523 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:10:15.518549 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518526 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:10:15.518549 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518529 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:10:15.518549 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518532 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:10:15.518549 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518534 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:10:15.518549 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518537 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:10:15.518549 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518540 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:10:15.518549 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518542 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:10:15.518549 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518545 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:10:15.518549 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518548 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:10:15.519002 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518551 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:10:15.519002 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518553 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:10:15.519002 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518556 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:10:15.519002 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518559 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:10:15.519002 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518562 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:10:15.519002 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518564 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:10:15.519002 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518568 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:10:15.519002 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518572 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:10:15.519002 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518574 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:10:15.519002 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518577 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:10:15.519002 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518580 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:10:15.519002 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518584 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:10:15.519002 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518587 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:10:15.519002 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518590 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:10:15.519002 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518593 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:10:15.519002 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518595 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:10:15.519002 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518598 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:10:15.519002 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518600 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:10:15.519002 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518603 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:10:15.519483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518605 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:10:15.519483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518608 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:10:15.519483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518610 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:10:15.519483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518612 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:10:15.519483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518615 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:10:15.519483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518618 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:10:15.519483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518620 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:10:15.519483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518623 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:10:15.519483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518625 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:10:15.519483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518628 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:10:15.519483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518631 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:10:15.519483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518633 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:10:15.519483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518636 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:10:15.519483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518638 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:10:15.519483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518642 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:10:15.519483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518645 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:10:15.519483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518647 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:10:15.519483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518650 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:10:15.519483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518653 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:10:15.519483 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518655 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:10:15.520004 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518658 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:10:15.520004 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518660 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:10:15.520004 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518663 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:10:15.520004 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518665 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:10:15.520004 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518674 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:10:15.520004 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518677 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:10:15.520004 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518679 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:10:15.520004 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518682 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:10:15.520004 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518684 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:10:15.520004 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518687 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:10:15.520004 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518689 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:10:15.520004 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518692 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:10:15.520004 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518694 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:10:15.520004 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518696 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:10:15.520004 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518699 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:10:15.520004 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518703 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:10:15.520004 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518706 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:10:15.520004 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518708 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:10:15.520004 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518711 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:10:15.520004 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518714 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:10:15.520508 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518719 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:10:15.520508 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518721 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:10:15.520508 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518724 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:10:15.520508 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518727 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:10:15.520508 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518730 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:10:15.520508 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518733 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:10:15.520508 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518737 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:10:15.520508 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518740 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:10:15.520508 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518742 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:10:15.520508 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518744 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:10:15.520508 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518747 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:10:15.520508 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518749 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:10:15.520508 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518752 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:10:15.520508 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518754 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:10:15.520508 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518757 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:10:15.520508 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518759 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:10:15.520508 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518762 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:10:15.520508 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518769 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:10:15.520508 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.518771 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:10:15.520508 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519552 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:10:15.520994 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519561 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:10:15.520994 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519565 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:10:15.520994 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519568 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:10:15.520994 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519571 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:10:15.520994 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519574 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:10:15.520994 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519577 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:10:15.520994 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519579 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:10:15.520994 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519582 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:10:15.520994 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519585 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:10:15.520994 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519587 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:10:15.520994 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519590 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:10:15.520994 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519593 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:10:15.520994 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519595 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:10:15.520994 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519598 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:10:15.520994 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519601 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:10:15.520994 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519604 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:10:15.520994 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519607 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:10:15.520994 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519609 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:10:15.520994 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519612 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:10:15.520994 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519615 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:10:15.521497 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519618 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:10:15.521497 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519621 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:10:15.521497 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519625 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:10:15.521497 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519628 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:10:15.521497 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519631 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:10:15.521497 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519633 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:10:15.521497 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519636 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:10:15.521497 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519638 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:10:15.521497 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519641 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:10:15.521497 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519644 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:10:15.521497 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519647 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:10:15.521497 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519650 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:10:15.521497 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519652 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:10:15.521497 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519655 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:10:15.521497 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519657 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:10:15.521497 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519662 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:10:15.521497 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519665 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:10:15.521497 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519668 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:10:15.521497 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519672 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:10:15.521995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519675 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:10:15.521995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519678 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:10:15.521995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519680 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:10:15.521995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519683 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:10:15.521995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519686 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:10:15.521995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519688 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:10:15.521995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519691 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:10:15.521995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519695 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:10:15.521995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519698 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:10:15.521995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519701 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:10:15.521995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519703 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:10:15.521995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519706 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:10:15.521995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519708 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:10:15.521995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519711 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:10:15.521995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519713 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:10:15.521995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519716 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:10:15.521995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519718 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:10:15.521995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519721 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:10:15.521995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519723 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:10:15.521995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519726 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:10:15.522514 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519729 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:10:15.522514 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519731 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:10:15.522514 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519734 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:10:15.522514 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519737 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:10:15.522514 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519741 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:10:15.522514 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519744 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:10:15.522514 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519747 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:10:15.522514 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519749 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:10:15.522514 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519753 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:10:15.522514 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519756 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:10:15.522514 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519760 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:10:15.522514 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519763 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:10:15.522514 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519766 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:10:15.522514 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519769 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:10:15.522514 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519772 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:10:15.522514 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519774 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:10:15.522514 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519776 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:10:15.522514 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519780 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:10:15.522514 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519782 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:10:15.523019 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519785 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:10:15.523019 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519788 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:10:15.523019 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519791 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:10:15.523019 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519793 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:10:15.523019 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519796 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:10:15.523019 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519798 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:10:15.523019 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519801 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:10:15.523019 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519804 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:10:15.523019 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519806 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:10:15.523019 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519810 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:10:15.523019 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519812 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:10:15.523019 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519815 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:10:15.523019 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519817 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:10:15.523019 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519820 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:10:15.523019 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519822 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:10:15.523019 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519825 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:10:15.523019 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519828 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:10:15.523019 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519831 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:10:15.523019 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519834 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:10:15.523019 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.519836 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:10:15.523525 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520227 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:10:15.523525 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520232 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:10:15.523525 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520235 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:10:15.523525 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520239 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:10:15.523525 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520242 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:10:15.523525 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520245 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:10:15.523525 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520248 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:10:15.523525 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520253 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:10:15.523525 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520257 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:10:15.523525 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520262 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:10:15.523525 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520265 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:10:15.523525 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520268 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:10:15.523525 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520271 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:10:15.523525 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520274 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:10:15.523525 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520277 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:10:15.523525 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520280 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:10:15.523525 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520283 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:10:15.523525 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520286 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:10:15.523525 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520289 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:10:15.523525 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520292 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:10:15.524008 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520295 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:10:15.524008 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520297 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:10:15.524008 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520301 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:10:15.524008 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520305 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:10:15.524008 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520308 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:10:15.524008 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520311 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:10:15.524008 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520313 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:10:15.524008 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520316 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:10:15.524008 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520318 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:10:15.524008 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520321 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:10:15.524008 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520324 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:10:15.524008 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520327 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:10:15.524008 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520329 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:10:15.524008 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520332 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:10:15.524008 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520335 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:10:15.524008 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520337 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:10:15.524008 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520340 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:10:15.524008 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520342 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:10:15.524008 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520345 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:10:15.524008 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520348 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:10:15.524512 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520350 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:10:15.524512 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520354 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:10:15.524512 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520357 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:10:15.524512 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520359 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:10:15.524512 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520362 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:10:15.524512 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520364 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:10:15.524512 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520367 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:10:15.524512 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520369 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:10:15.524512 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520373 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:10:15.524512 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520375 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:10:15.524512 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520378 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:10:15.524512 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520381 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:10:15.524512 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520383 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:10:15.524512 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520401 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:10:15.524512 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520403 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:10:15.524512 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520406 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:10:15.524512 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520408 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:10:15.524512 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520411 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:10:15.524512 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520414 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:10:15.524512 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520416 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:10:15.524995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520419 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:10:15.524995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520422 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:10:15.524995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520425 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:10:15.524995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520428 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:10:15.524995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520430 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:10:15.524995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520433 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:10:15.524995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520436 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:10:15.524995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520438 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:10:15.524995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520441 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:10:15.524995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520443 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:10:15.524995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520446 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:10:15.524995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520448 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:10:15.524995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520451 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:10:15.524995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520456 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:10:15.524995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520460 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:10:15.524995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520464 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:10:15.524995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520467 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:10:15.524995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520470 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:10:15.524995 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520473 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:10:15.525475 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520476 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:10:15.525475 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520479 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:10:15.525475 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520483 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:10:15.525475 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520485 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:10:15.525475 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520488 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:10:15.525475 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520492 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:10:15.525475 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520495 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:10:15.525475 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520498 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:10:15.525475 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520501 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:10:15.525475 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520504 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:10:15.525475 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520507 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:10:15.525475 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520510 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:10:15.525475 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520514 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:10:15.525475 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520516 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:10:15.525475 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520519 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:10:15.525475 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520522 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:10:15.525475 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520525 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:10:15.525475 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520528 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:10:15.525475 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520530 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:10:15.525475 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.520533 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:10:15.525475 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520602 2550 flags.go:64] FLAG: --address="0.0.0.0" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520610 2550 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520616 2550 flags.go:64] FLAG: --anonymous-auth="true" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520620 2550 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520625 2550 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520628 2550 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520634 2550 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520638 2550 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520641 2550 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520644 2550 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520648 2550 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520651 2550 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520655 2550 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520658 2550 flags.go:64] FLAG: --cgroup-root="" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520661 2550 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520664 2550 flags.go:64] FLAG: --client-ca-file="" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520667 2550 flags.go:64] FLAG: --cloud-provider="external" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520671 2550 flags.go:64] FLAG: --cluster-dns="[]" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520676 2550 flags.go:64] FLAG: --cluster-domain="" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520679 2550 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520682 2550 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520685 2550 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520689 2550 flags.go:64] FLAG: --container-log-max-files="5" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520693 2550 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 28 17:10:15.526012 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520696 2550 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520699 2550 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520702 2550 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520706 2550 flags.go:64] FLAG: --contention-profiling="false" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520709 2550 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520712 2550 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520716 2550 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520719 2550 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520723 2550 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520726 2550 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520729 2550 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520732 2550 flags.go:64] FLAG: --enable-load-reader="false" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520735 2550 flags.go:64] FLAG: --enable-server="true" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520738 2550 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520744 2550 flags.go:64] FLAG: --event-burst="100" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520747 2550 flags.go:64] FLAG: --event-qps="50" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520750 2550 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520753 2550 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520756 2550 flags.go:64] FLAG: --eviction-hard="" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520760 2550 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520762 2550 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520766 2550 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520769 2550 flags.go:64] FLAG: --eviction-soft="" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520772 2550 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520776 2550 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 28 17:10:15.526625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520779 2550 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520782 2550 flags.go:64] FLAG: --experimental-mounter-path="" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520785 2550 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520788 2550 flags.go:64] FLAG: --fail-swap-on="true" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520791 2550 flags.go:64] FLAG: --feature-gates="" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520795 2550 flags.go:64] FLAG: --file-check-frequency="20s" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520798 2550 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520801 2550 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520805 2550 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520808 2550 flags.go:64] FLAG: --healthz-port="10248" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520811 2550 flags.go:64] FLAG: --help="false" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520814 2550 flags.go:64] FLAG: --hostname-override="ip-10-0-134-14.ec2.internal" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520817 2550 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520820 2550 flags.go:64] FLAG: --http-check-frequency="20s" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520824 2550 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520827 2550 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520830 2550 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520833 2550 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520836 2550 flags.go:64] FLAG: --image-service-endpoint="" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520839 2550 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520841 2550 flags.go:64] FLAG: --kube-api-burst="100" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520847 2550 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520851 2550 flags.go:64] FLAG: --kube-api-qps="50" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520853 2550 flags.go:64] FLAG: --kube-reserved="" Jul 28 17:10:15.527273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520856 2550 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520859 2550 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520862 2550 flags.go:64] FLAG: --kubelet-cgroups="" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520865 2550 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520868 2550 flags.go:64] FLAG: --lock-file="" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520870 2550 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520873 2550 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520878 2550 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520883 2550 flags.go:64] FLAG: --log-json-split-stream="false" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520886 2550 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520889 2550 flags.go:64] FLAG: --log-text-split-stream="false" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520892 2550 flags.go:64] FLAG: --logging-format="text" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520894 2550 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520898 2550 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520901 2550 flags.go:64] FLAG: --manifest-url="" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520903 2550 flags.go:64] FLAG: --manifest-url-header="" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520907 2550 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520910 2550 flags.go:64] FLAG: --max-open-files="1000000" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520915 2550 flags.go:64] FLAG: --max-pods="110" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520918 2550 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520921 2550 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520924 2550 flags.go:64] FLAG: --memory-manager-policy="None" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520927 2550 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520931 2550 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520934 2550 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 28 17:10:15.527891 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520937 2550 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520944 2550 flags.go:64] FLAG: --node-status-max-images="50" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520947 2550 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520951 2550 flags.go:64] FLAG: --oom-score-adj="-999" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520955 2550 flags.go:64] FLAG: --pod-cidr="" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520958 2550 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520964 2550 flags.go:64] FLAG: --pod-manifest-path="" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520967 2550 flags.go:64] FLAG: --pod-max-pids="-1" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520970 2550 flags.go:64] FLAG: --pods-per-core="0" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520973 2550 flags.go:64] FLAG: --port="10250" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520976 2550 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520979 2550 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-069f424c0c1191d79" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520982 2550 flags.go:64] FLAG: --qos-reserved="" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520985 2550 flags.go:64] FLAG: --read-only-port="10255" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520989 2550 flags.go:64] FLAG: --register-node="true" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520992 2550 flags.go:64] FLAG: --register-with-taints="" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520996 2550 flags.go:64] FLAG: --registry-burst="10" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.520999 2550 flags.go:64] FLAG: --registry-qps="5" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521003 2550 flags.go:64] FLAG: --reserved-cpus="" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521006 2550 flags.go:64] FLAG: --reserved-memory="" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521009 2550 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521013 2550 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521016 2550 flags.go:64] FLAG: --rotate-certificates="false" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521019 2550 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521022 2550 flags.go:64] FLAG: --runonce="false" Jul 28 17:10:15.528510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521024 2550 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521028 2550 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521031 2550 flags.go:64] FLAG: --seccomp-default="false" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521034 2550 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521037 2550 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521040 2550 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521043 2550 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521046 2550 flags.go:64] FLAG: --storage-driver-password="root" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521049 2550 flags.go:64] FLAG: --storage-driver-secure="false" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521052 2550 flags.go:64] FLAG: --storage-driver-table="stats" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521055 2550 flags.go:64] FLAG: --storage-driver-user="root" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521058 2550 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521062 2550 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521065 2550 flags.go:64] FLAG: --system-cgroups="" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521068 2550 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521074 2550 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521076 2550 flags.go:64] FLAG: --tls-cert-file="" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521079 2550 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521084 2550 flags.go:64] FLAG: --tls-min-version="" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521087 2550 flags.go:64] FLAG: --tls-private-key-file="" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521089 2550 flags.go:64] FLAG: --topology-manager-policy="none" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521094 2550 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521097 2550 flags.go:64] FLAG: --topology-manager-scope="container" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521100 2550 flags.go:64] FLAG: --v="2" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521104 2550 flags.go:64] FLAG: --version="false" Jul 28 17:10:15.529123 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521108 2550 flags.go:64] FLAG: --vmodule="" Jul 28 17:10:15.529746 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521112 2550 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 28 17:10:15.529746 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.521115 2550 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 28 17:10:15.529746 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521234 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:10:15.529746 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521238 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:10:15.529746 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521241 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:10:15.529746 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521243 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:10:15.529746 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521246 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:10:15.529746 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521248 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:10:15.529746 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521251 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:10:15.529746 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521254 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:10:15.529746 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521256 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:10:15.529746 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521259 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:10:15.529746 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521262 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:10:15.529746 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521265 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:10:15.529746 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521268 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:10:15.529746 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521270 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:10:15.529746 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521273 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:10:15.529746 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521276 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:10:15.529746 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521279 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:10:15.529746 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521281 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:10:15.530241 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521284 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:10:15.530241 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521287 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:10:15.530241 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521289 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:10:15.530241 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521292 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:10:15.530241 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521295 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:10:15.530241 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521297 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:10:15.530241 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521300 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:10:15.530241 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521304 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:10:15.530241 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521307 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:10:15.530241 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521311 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:10:15.530241 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521314 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:10:15.530241 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521317 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:10:15.530241 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521320 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:10:15.530241 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521323 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:10:15.530241 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521326 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:10:15.530241 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521329 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:10:15.530241 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521332 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:10:15.530241 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521335 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:10:15.530241 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521337 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:10:15.530241 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521340 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:10:15.530809 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521342 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:10:15.530809 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521345 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:10:15.530809 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521347 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:10:15.530809 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521350 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:10:15.530809 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521353 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:10:15.530809 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521355 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:10:15.530809 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521359 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:10:15.530809 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521362 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:10:15.530809 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521364 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:10:15.530809 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521367 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:10:15.530809 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521369 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:10:15.530809 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521372 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:10:15.530809 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521375 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:10:15.530809 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521378 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:10:15.530809 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521381 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:10:15.530809 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521397 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:10:15.530809 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521400 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:10:15.530809 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521403 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:10:15.530809 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521405 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:10:15.530809 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521409 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:10:15.531610 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521412 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:10:15.531610 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521415 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:10:15.531610 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521417 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:10:15.531610 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521420 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:10:15.531610 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521422 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:10:15.531610 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521425 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:10:15.531610 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521427 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:10:15.531610 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521430 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:10:15.531610 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521432 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:10:15.531610 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521435 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:10:15.531610 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521438 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:10:15.531610 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521440 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:10:15.531610 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521443 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:10:15.531610 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521445 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:10:15.531610 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521448 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:10:15.531610 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521450 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:10:15.531610 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521453 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:10:15.531610 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521456 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:10:15.531610 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521458 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:10:15.531610 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521461 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:10:15.532299 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521465 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:10:15.532299 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521469 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:10:15.532299 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521472 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:10:15.532299 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521475 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:10:15.532299 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521478 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:10:15.532299 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521481 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:10:15.532299 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521484 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:10:15.532299 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521487 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:10:15.532299 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521490 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:10:15.532299 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521492 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:10:15.532299 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521495 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:10:15.532299 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521499 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:10:15.532299 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521501 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:10:15.532299 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521504 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:10:15.532299 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521507 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:10:15.532299 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521510 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:10:15.532299 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521512 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:10:15.532299 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521515 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:10:15.532299 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521518 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:10:15.532790 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521521 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:10:15.532790 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.521523 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:10:15.532790 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.522283 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 17:10:15.532790 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.530176 2550 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 28 17:10:15.532790 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.530196 2550 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 28 17:10:15.532790 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.530220 2550 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 28 17:10:15.532790 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.530228 2550 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 28 17:10:15.532790 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530285 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:10:15.532790 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530290 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:10:15.532790 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530294 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:10:15.532790 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530298 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:10:15.532790 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530302 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:10:15.532790 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530304 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:10:15.532790 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530308 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:10:15.533150 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530311 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:10:15.533150 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530314 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:10:15.533150 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530317 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:10:15.533150 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530320 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:10:15.533150 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530322 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:10:15.533150 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530325 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:10:15.533150 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530328 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:10:15.533150 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530331 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:10:15.533150 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530333 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:10:15.533150 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530336 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:10:15.533150 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530338 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:10:15.533150 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530341 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:10:15.533150 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530343 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:10:15.533150 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530346 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:10:15.533150 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530348 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:10:15.533150 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530351 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:10:15.533150 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530354 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:10:15.533150 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530357 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:10:15.533150 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530360 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:10:15.533150 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530362 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:10:15.533656 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530365 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:10:15.533656 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530368 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:10:15.533656 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530373 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:10:15.533656 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530378 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:10:15.533656 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530381 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:10:15.533656 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530384 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:10:15.533656 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530404 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:10:15.533656 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530408 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:10:15.533656 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530412 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:10:15.533656 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530416 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:10:15.533656 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530421 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:10:15.533656 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530424 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:10:15.533656 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530426 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:10:15.533656 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530429 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:10:15.533656 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530432 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:10:15.533656 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530434 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:10:15.533656 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530437 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:10:15.533656 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530440 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:10:15.533656 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530442 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:10:15.533656 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530445 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:10:15.534154 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530448 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:10:15.534154 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530451 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:10:15.534154 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530453 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:10:15.534154 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530456 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:10:15.534154 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530459 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:10:15.534154 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530461 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:10:15.534154 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530464 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:10:15.534154 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530467 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:10:15.534154 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530469 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:10:15.534154 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530472 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:10:15.534154 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530475 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:10:15.534154 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530477 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:10:15.534154 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530480 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:10:15.534154 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530483 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:10:15.534154 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530487 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:10:15.534154 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530490 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:10:15.534154 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530493 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:10:15.534154 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530495 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:10:15.534154 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530498 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:10:15.534154 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530501 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:10:15.534718 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530503 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:10:15.534718 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530683 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:10:15.534718 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530695 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:10:15.534718 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530702 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:10:15.534718 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530708 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:10:15.534718 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530714 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:10:15.534718 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530719 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:10:15.534718 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530724 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:10:15.534718 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530729 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:10:15.534718 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530734 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:10:15.534718 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530739 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:10:15.534718 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530744 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:10:15.534718 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530749 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:10:15.534718 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530761 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:10:15.534718 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530765 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:10:15.534718 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530770 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:10:15.534718 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530774 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:10:15.534718 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530781 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:10:15.534718 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530786 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:10:15.535184 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530791 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:10:15.535184 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530796 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:10:15.535184 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530801 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:10:15.535184 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530805 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:10:15.535184 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530810 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:10:15.535184 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530815 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:10:15.535184 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530825 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:10:15.535184 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530830 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:10:15.535184 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530835 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:10:15.535184 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530851 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:10:15.535184 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530856 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:10:15.535184 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530860 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:10:15.535184 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.530864 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:10:15.535184 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.530873 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 17:10:15.535184 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531300 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 28 17:10:15.535727 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531309 2550 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 28 17:10:15.535727 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531314 2550 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 28 17:10:15.535727 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531320 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 28 17:10:15.535727 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531324 2550 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 28 17:10:15.535727 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531329 2550 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 28 17:10:15.535727 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531333 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 28 17:10:15.535727 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531338 2550 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 28 17:10:15.535727 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531342 2550 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 28 17:10:15.535727 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531352 2550 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 28 17:10:15.535727 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531356 2550 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 28 17:10:15.535727 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531360 2550 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 28 17:10:15.535727 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531364 2550 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 28 17:10:15.535727 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531369 2550 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 28 17:10:15.535727 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531373 2550 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 28 17:10:15.535727 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531377 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 28 17:10:15.535727 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531381 2550 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 28 17:10:15.535727 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531404 2550 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 28 17:10:15.535727 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531409 2550 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 28 17:10:15.535727 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531413 2550 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 28 17:10:15.535727 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531417 2550 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 28 17:10:15.536327 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531426 2550 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 28 17:10:15.536327 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531430 2550 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 28 17:10:15.536327 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531435 2550 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 28 17:10:15.536327 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531439 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 28 17:10:15.536327 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531443 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 28 17:10:15.536327 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531447 2550 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 28 17:10:15.536327 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531451 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 28 17:10:15.536327 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531456 2550 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 28 17:10:15.536327 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531461 2550 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 28 17:10:15.536327 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531465 2550 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 28 17:10:15.536327 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531470 2550 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 28 17:10:15.536327 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531474 2550 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 28 17:10:15.536327 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531478 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 28 17:10:15.536327 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531487 2550 feature_gate.go:328] unrecognized feature gate: Example Jul 28 17:10:15.536327 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531492 2550 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 28 17:10:15.536327 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531497 2550 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 28 17:10:15.536327 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531501 2550 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 28 17:10:15.536327 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531506 2550 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 28 17:10:15.536327 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531511 2550 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 28 17:10:15.536327 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531516 2550 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 28 17:10:15.536877 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531520 2550 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 28 17:10:15.536877 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531525 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 28 17:10:15.536877 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531529 2550 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 28 17:10:15.536877 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531533 2550 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 28 17:10:15.536877 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531538 2550 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 28 17:10:15.536877 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531545 2550 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 28 17:10:15.536877 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531556 2550 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 28 17:10:15.536877 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531561 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 28 17:10:15.536877 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531567 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 28 17:10:15.536877 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531573 2550 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 28 17:10:15.536877 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531578 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 28 17:10:15.536877 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531591 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 28 17:10:15.536877 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531596 2550 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 28 17:10:15.536877 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531601 2550 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 28 17:10:15.536877 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531604 2550 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 28 17:10:15.536877 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531607 2550 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 28 17:10:15.536877 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531609 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 28 17:10:15.536877 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531612 2550 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 28 17:10:15.536877 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531618 2550 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 28 17:10:15.537448 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531621 2550 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 28 17:10:15.537448 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531627 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 28 17:10:15.537448 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531631 2550 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 28 17:10:15.537448 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531633 2550 feature_gate.go:328] unrecognized feature gate: Example2 Jul 28 17:10:15.537448 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531637 2550 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 28 17:10:15.537448 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531642 2550 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 28 17:10:15.537448 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531646 2550 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 28 17:10:15.537448 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531651 2550 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 28 17:10:15.537448 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531655 2550 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 28 17:10:15.537448 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531662 2550 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 28 17:10:15.537448 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531667 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 28 17:10:15.537448 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531672 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 28 17:10:15.537448 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531677 2550 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 28 17:10:15.537448 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531682 2550 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 28 17:10:15.537448 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531692 2550 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 28 17:10:15.537448 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531711 2550 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 28 17:10:15.537448 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531715 2550 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 28 17:10:15.537448 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531816 2550 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 28 17:10:15.537448 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531820 2550 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 28 17:10:15.537448 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531823 2550 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 28 17:10:15.537953 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531826 2550 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 28 17:10:15.537953 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531829 2550 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 28 17:10:15.537953 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531832 2550 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 28 17:10:15.537953 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531835 2550 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 28 17:10:15.537953 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531838 2550 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 28 17:10:15.537953 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531840 2550 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 28 17:10:15.537953 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531843 2550 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 28 17:10:15.537953 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531848 2550 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 28 17:10:15.537953 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531852 2550 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 28 17:10:15.537953 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531855 2550 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 28 17:10:15.537953 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531858 2550 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 28 17:10:15.537953 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531860 2550 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 28 17:10:15.537953 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531863 2550 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 28 17:10:15.537953 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531866 2550 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 28 17:10:15.537953 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531870 2550 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 28 17:10:15.537953 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531872 2550 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 28 17:10:15.537953 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531875 2550 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 28 17:10:15.537953 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531879 2550 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 28 17:10:15.537953 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:15.531881 2550 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 28 17:10:15.538679 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.531888 2550 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 28 17:10:15.538679 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.532024 2550 server.go:962] "Client rotation is on, will bootstrap in background" Jul 28 17:10:15.538679 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.534223 2550 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 28 17:10:15.538679 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.535692 2550 server.go:1019] "Starting client certificate rotation" Jul 28 17:10:15.538679 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.535815 2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 17:10:15.538679 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.535861 2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 28 17:10:15.538939 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.538746 2550 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 17:10:15.540609 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.540586 2550 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 28 17:10:15.549524 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.549507 2550 log.go:25] "Validated CRI v1 runtime API" Jul 28 17:10:15.553717 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.553699 2550 log.go:25] "Validated CRI v1 image API" Jul 28 17:10:15.554508 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.554491 2550 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 28 17:10:15.556983 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.556960 2550 fs.go:135] Filesystem UUIDs: map[0b8468e9-a10c-4676-a44b-fb7bf398463f:/dev/nvme0n1p3 6ff887e9-05e2-4703-83f6-3ecb5fe3917f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 28 17:10:15.557048 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.556982 2550 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 28 17:10:15.562105 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.561989 2550 manager.go:216] Machine: {Timestamp:2026-07-28 17:10:15.561526712 +0000 UTC m=+0.290028039 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3096485 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27a35712d0926921ec6c56add90916 SystemUUID:ec27a357-12d0-9269-21ec-6c56add90916 BootID:4af9c897-5305-46ea-ae73-3817043cda4c Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b5:08:e2:61:63 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b5:08:e2:61:63 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:af:26:89:92:b5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 28 17:10:15.562105 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.562103 2550 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 28 17:10:15.562215 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.562186 2550 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 28 17:10:15.563145 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.563119 2550 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 28 17:10:15.563289 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.563153 2550 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-14.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 28 17:10:15.563812 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.563802 2550 topology_manager.go:138] "Creating topology manager with none policy" Jul 28 17:10:15.563845 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.563815 2550 container_manager_linux.go:309] "Creating device plugin manager" Jul 28 17:10:15.563845 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.563822 2550 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 17:10:15.563845 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.563834 2550 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 28 17:10:15.565368 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.565358 2550 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 28 17:10:15.567948 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.567935 2550 state_mem.go:36] "Initialized new in-memory state store" Jul 28 17:10:15.568066 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.568056 2550 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 28 17:10:15.568711 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.568702 2550 kubelet.go:486] "Attempting to sync node with API server" Jul 28 17:10:15.568742 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.568715 2550 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 28 17:10:15.568742 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.568726 2550 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 28 17:10:15.568742 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.568736 2550 kubelet.go:398] "Adding apiserver pod source" Jul 28 17:10:15.568844 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.568745 2550 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 28 17:10:15.569181 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.569161 2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 17:10:15.569281 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.569196 2550 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 28 17:10:15.571969 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.571945 2550 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 28 17:10:15.573114 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.573097 2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 17:10:15.574113 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.574097 2550 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 28 17:10:15.574177 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.574129 2550 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 28 17:10:15.575641 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.575629 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 28 17:10:15.575702 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.575645 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 28 17:10:15.575702 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.575654 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 28 17:10:15.575702 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.575662 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 28 17:10:15.575702 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.575675 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 28 17:10:15.575702 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.575684 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 28 17:10:15.575702 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.575696 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 28 17:10:15.575702 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.575703 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 28 17:10:15.575880 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.575710 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 28 17:10:15.575880 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.575717 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 28 17:10:15.575880 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.575726 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 28 17:10:15.575880 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.575731 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 28 17:10:15.576355 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.576344 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 28 17:10:15.576383 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.576355 2550 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 28 17:10:15.579462 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.579439 2550 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-14.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 28 17:10:15.579567 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.579522 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 28 17:10:15.579567 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.579541 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-14.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 28 17:10:15.579859 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.579845 2550 server.go:1268] "Started kubelet" Jul 28 17:10:15.579935 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.579900 2550 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 28 17:10:15.580062 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.579985 2550 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 28 17:10:15.580119 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.580076 2550 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 28 17:10:15.580752 ip-10-0-134-14 systemd[1]: Started Kubernetes Kubelet. Jul 28 17:10:15.581790 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.581503 2550 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 28 17:10:15.582510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.582303 2550 server.go:310] "Adding debug handlers to kubelet server" Jul 28 17:10:15.587351 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.587321 2550 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 28 17:10:15.588543 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.588527 2550 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 28 17:10:15.588543 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.588539 2550 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 28 17:10:15.589191 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.588604 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-14.ec2.internal\" not found" Jul 28 17:10:15.589191 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.589175 2550 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 28 17:10:15.589191 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.589177 2550 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 28 17:10:15.590040 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.589202 2550 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 28 17:10:15.590040 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.589282 2550 reconstruct.go:97] "Volume reconstruction finished" Jul 28 17:10:15.590040 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.589293 2550 reconciler.go:29] "Reconciler: start to sync state" Jul 28 17:10:15.590040 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.589561 2550 factory.go:55] Registering systemd factory Jul 28 17:10:15.590040 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.589574 2550 factory.go:223] Registration of the systemd container factory successfully Jul 28 17:10:15.590040 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.588946 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-14.ec2.internal.18c683408dd72e83 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-14.ec2.internal,UID:ip-10-0-134-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-14.ec2.internal,},FirstTimestamp:2026-07-28 17:10:15.579815555 +0000 UTC m=+0.308316883,LastTimestamp:2026-07-28 17:10:15.579815555 +0000 UTC m=+0.308316883,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-14.ec2.internal,}" Jul 28 17:10:15.590040 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.589775 2550 factory.go:153] Registering CRI-O factory Jul 28 17:10:15.590040 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.589785 2550 factory.go:223] Registration of the crio container factory successfully Jul 28 17:10:15.590040 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.589840 2550 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 28 17:10:15.590040 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.589863 2550 factory.go:103] Registering Raw factory Jul 28 17:10:15.590040 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.589876 2550 manager.go:1196] Started watching for new ooms in manager Jul 28 17:10:15.590627 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.590463 2550 manager.go:349] Starting recovery of all containers Jul 28 17:10:15.599049 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.598708 2550 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-14.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 28 17:10:15.599049 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.598848 2550 manager.go:354] Recovery completed Jul 28 17:10:15.599187 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.599077 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 28 17:10:15.605710 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.605694 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:10:15.608088 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.608060 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:10:15.608150 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.608110 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:10:15.608150 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.608124 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:10:15.608778 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.608764 2550 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 28 17:10:15.608778 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.608778 2550 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 28 17:10:15.608897 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.608784 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:10:15.608897 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.608793 2550 state_mem.go:36] "Initialized new in-memory state store" Jul 28 17:10:15.609581 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.609562 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:10:15.609640 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.609597 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:10:15.609640 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.609612 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:10:15.609948 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.609882 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-14.ec2.internal.18c683408f869b1e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-14.ec2.internal,UID:ip-10-0-134-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-14.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-14.ec2.internal,},FirstTimestamp:2026-07-28 17:10:15.608089374 +0000 UTC m=+0.336590709,LastTimestamp:2026-07-28 17:10:15.608089374 +0000 UTC m=+0.336590709,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-14.ec2.internal,}" Jul 28 17:10:15.610296 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.610285 2550 policy_none.go:49] "None policy: Start" Jul 28 17:10:15.610330 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.610306 2550 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 28 17:10:15.610330 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.610317 2550 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 28 17:10:15.611353 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.611341 2550 policy_none.go:47] "Start" Jul 28 17:10:15.623322 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.623251 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-14.ec2.internal.18c683408f87048f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-14.ec2.internal,UID:ip-10-0-134-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-14.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-14.ec2.internal,},FirstTimestamp:2026-07-28 17:10:15.608116367 +0000 UTC m=+0.336617694,LastTimestamp:2026-07-28 17:10:15.608116367 +0000 UTC m=+0.336617694,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-14.ec2.internal,}" Jul 28 17:10:15.637508 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.637427 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-14.ec2.internal.18c683408f8738a9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-14.ec2.internal,UID:ip-10-0-134-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-14.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-14.ec2.internal,},FirstTimestamp:2026-07-28 17:10:15.608129705 +0000 UTC m=+0.336631035,LastTimestamp:2026-07-28 17:10:15.608129705 +0000 UTC m=+0.336631035,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-14.ec2.internal,}" Jul 28 17:10:15.644106 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.644087 2550 manager.go:337] "Starting Device Plugin manager" Jul 28 17:10:15.644223 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.644129 2550 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 28 17:10:15.644223 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.644143 2550 server.go:87] "Starting device plugin registration server" Jul 28 17:10:15.644445 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.644431 2550 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 28 17:10:15.644499 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.644447 2550 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 28 17:10:15.644744 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.644603 2550 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 28 17:10:15.644744 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.644689 2550 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 28 17:10:15.644744 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.644698 2550 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 28 17:10:15.645172 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.645155 2550 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 28 17:10:15.645257 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.645201 2550 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-14.ec2.internal\" not found" Jul 28 17:10:15.647467 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.647382 2550 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-14.ec2.internal.18c683408f869b1e\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-14.ec2.internal.18c683408f869b1e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-14.ec2.internal,UID:ip-10-0-134-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-14.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-14.ec2.internal,},FirstTimestamp:2026-07-28 17:10:15.608089374 +0000 UTC m=+0.336590709,LastTimestamp:2026-07-28 17:10:15.609580098 +0000 UTC m=+0.338081428,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-14.ec2.internal,}" Jul 28 17:10:15.659092 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.659020 2550 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-14.ec2.internal.18c683408f87048f\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-14.ec2.internal.18c683408f87048f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-14.ec2.internal,UID:ip-10-0-134-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-14.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-14.ec2.internal,},FirstTimestamp:2026-07-28 17:10:15.608116367 +0000 UTC m=+0.336617694,LastTimestamp:2026-07-28 17:10:15.609603988 +0000 UTC m=+0.338105316,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-14.ec2.internal,}" Jul 28 17:10:15.660077 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.660057 2550 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2583 into the system container \"/system.slice\": " Jul 28 17:10:15.661656 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.661639 2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gq8lc" Jul 28 17:10:15.671762 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.671673 2550 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-14.ec2.internal.18c683408f8738a9\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-14.ec2.internal.18c683408f8738a9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-14.ec2.internal,UID:ip-10-0-134-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-14.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-14.ec2.internal,},FirstTimestamp:2026-07-28 17:10:15.608129705 +0000 UTC m=+0.336631035,LastTimestamp:2026-07-28 17:10:15.6096181 +0000 UTC m=+0.338119429,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-14.ec2.internal,}" Jul 28 17:10:15.682553 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.676311 2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-gq8lc" Jul 28 17:10:15.682787 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.682716 2550 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-14.ec2.internal.18c68340925067b8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-14.ec2.internal,UID:ip-10-0-134-14.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-134-14.ec2.internal,},FirstTimestamp:2026-07-28 17:10:15.65486892 +0000 UTC m=+0.383370235,LastTimestamp:2026-07-28 17:10:15.65486892 +0000 UTC m=+0.383370235,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-14.ec2.internal,}" Jul 28 17:10:15.745195 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.745155 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:10:15.746219 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.746195 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:10:15.746339 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.746261 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:10:15.746339 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.746290 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:10:15.746339 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.746317 2550 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-14.ec2.internal" Jul 28 17:10:15.764471 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.764378 2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 28 17:10:15.765672 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.765646 2550 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 28 17:10:15.765792 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.765691 2550 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 28 17:10:15.765792 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.765718 2550 kubelet.go:2448] "Starting kubelet main sync loop" Jul 28 17:10:15.765792 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.765758 2550 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 28 17:10:15.767324 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.767299 2550 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-14.ec2.internal" Jul 28 17:10:15.767459 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.767335 2550 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-14.ec2.internal\": node \"ip-10-0-134-14.ec2.internal\" not found" Jul 28 17:10:15.768698 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.768550 2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:10:15.837429 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.837384 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-14.ec2.internal\" not found" Jul 28 17:10:15.866492 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.866447 2550 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-14.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-14.ec2.internal"] Jul 28 17:10:15.866569 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.866548 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:10:15.867504 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.867487 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:10:15.867573 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.867524 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:10:15.867573 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.867539 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:10:15.868709 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.868697 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:10:15.868812 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.868799 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-14.ec2.internal" Jul 28 17:10:15.868852 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.868829 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:10:15.869402 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.869366 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:10:15.869503 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.869413 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:10:15.869503 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.869427 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:10:15.869503 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.869445 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:10:15.869503 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.869465 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:10:15.869503 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.869475 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:10:15.870551 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.870534 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-14.ec2.internal" Jul 28 17:10:15.870645 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.870565 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:10:15.871211 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.871197 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:10:15.871280 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.871223 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:10:15.871280 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.871237 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:10:15.891449 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.891423 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7cc46c224c17195fb5410e94fda8e4a3-config\") pod \"kube-apiserver-proxy-ip-10-0-134-14.ec2.internal\" (UID: \"7cc46c224c17195fb5410e94fda8e4a3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-14.ec2.internal" Jul 28 17:10:15.891449 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.891446 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1b52734c50b11119e9322d7d16760bf3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-14.ec2.internal\" (UID: \"1b52734c50b11119e9322d7d16760bf3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-14.ec2.internal" Jul 28 17:10:15.891566 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.891463 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b52734c50b11119e9322d7d16760bf3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-14.ec2.internal\" (UID: \"1b52734c50b11119e9322d7d16760bf3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-14.ec2.internal" Jul 28 17:10:15.897133 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.897113 2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-14.ec2.internal\" not found" node="ip-10-0-134-14.ec2.internal" Jul 28 17:10:15.900735 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.900719 2550 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-14.ec2.internal\" not found" node="ip-10-0-134-14.ec2.internal" Jul 28 17:10:15.938331 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:15.938294 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-14.ec2.internal\" not found" Jul 28 17:10:15.991835 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.991806 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1b52734c50b11119e9322d7d16760bf3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-14.ec2.internal\" (UID: \"1b52734c50b11119e9322d7d16760bf3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-14.ec2.internal" Jul 28 17:10:15.991936 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.991833 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1b52734c50b11119e9322d7d16760bf3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-14.ec2.internal\" (UID: \"1b52734c50b11119e9322d7d16760bf3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-14.ec2.internal" Jul 28 17:10:15.991936 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.991899 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b52734c50b11119e9322d7d16760bf3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-14.ec2.internal\" (UID: \"1b52734c50b11119e9322d7d16760bf3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-14.ec2.internal" Jul 28 17:10:15.991936 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.991930 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7cc46c224c17195fb5410e94fda8e4a3-config\") pod \"kube-apiserver-proxy-ip-10-0-134-14.ec2.internal\" (UID: \"7cc46c224c17195fb5410e94fda8e4a3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-14.ec2.internal" Jul 28 17:10:15.992061 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.991914 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b52734c50b11119e9322d7d16760bf3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-14.ec2.internal\" (UID: \"1b52734c50b11119e9322d7d16760bf3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-14.ec2.internal" Jul 28 17:10:15.992061 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:15.991967 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7cc46c224c17195fb5410e94fda8e4a3-config\") pod \"kube-apiserver-proxy-ip-10-0-134-14.ec2.internal\" (UID: \"7cc46c224c17195fb5410e94fda8e4a3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-14.ec2.internal" Jul 28 17:10:16.038842 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:16.038810 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-14.ec2.internal\" not found" Jul 28 17:10:16.139636 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:16.139589 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-14.ec2.internal\" not found" Jul 28 17:10:16.199133 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:16.199110 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-14.ec2.internal" Jul 28 17:10:16.205049 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:16.205028 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-14.ec2.internal" Jul 28 17:10:16.240476 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:16.240441 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-14.ec2.internal\" not found" Jul 28 17:10:16.341040 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:16.340938 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-14.ec2.internal\" not found" Jul 28 17:10:16.441548 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:16.441510 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-14.ec2.internal\" not found" Jul 28 17:10:16.522036 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:16.522012 2550 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:10:16.535018 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:16.534989 2550 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 28 17:10:16.535189 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:16.535159 2550 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 28 17:10:16.535257 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:16.535171 2550 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 28 17:10:16.541749 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:16.541723 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-14.ec2.internal\" not found" Jul 28 17:10:16.642451 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:16.642426 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-14.ec2.internal\" not found" Jul 28 17:10:16.681936 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:16.681873 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 17:05:16 +0000 UTC" deadline="2028-02-21 11:28:20.904982457 +0000 UTC" Jul 28 17:10:16.681936 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:16.681930 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13746h18m4.223057199s" Jul 28 17:10:16.704701 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:16.704673 2550 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 28 17:10:16.707757 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:16.706820 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasSufficientMemory" Jul 28 17:10:16.707757 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:16.706865 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasNoDiskPressure" Jul 28 17:10:16.707757 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:16.706881 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeHasSufficientPID" Jul 28 17:10:16.743580 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:16.743539 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-14.ec2.internal\" not found" Jul 28 17:10:16.760115 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:16.760080 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cc46c224c17195fb5410e94fda8e4a3.slice/crio-b72d788ac6abda63a8453a430b5cc5195f103912e10a82540b450be50cac9a63 WatchSource:0}: Error finding container b72d788ac6abda63a8453a430b5cc5195f103912e10a82540b450be50cac9a63: Status 404 returned error can't find the container with id b72d788ac6abda63a8453a430b5cc5195f103912e10a82540b450be50cac9a63 Jul 28 17:10:16.760377 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:16.760356 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b52734c50b11119e9322d7d16760bf3.slice/crio-c23782e9b736b86133c5ae28888db19fc76687958bcea47172008a99cb77e157 WatchSource:0}: Error finding container c23782e9b736b86133c5ae28888db19fc76687958bcea47172008a99cb77e157: Status 404 returned error can't find the container with id c23782e9b736b86133c5ae28888db19fc76687958bcea47172008a99cb77e157 Jul 28 17:10:16.764121 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:16.764107 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:10:16.768901 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:16.768854 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-14.ec2.internal" event={"ID":"7cc46c224c17195fb5410e94fda8e4a3","Type":"ContainerStarted","Data":"b72d788ac6abda63a8453a430b5cc5195f103912e10a82540b450be50cac9a63"} Jul 28 17:10:16.775845 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:16.775818 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-14.ec2.internal" event={"ID":"1b52734c50b11119e9322d7d16760bf3","Type":"ContainerStarted","Data":"c23782e9b736b86133c5ae28888db19fc76687958bcea47172008a99cb77e157"} Jul 28 17:10:16.845348 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:16.845304 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-14.ec2.internal\" not found" Jul 28 17:10:16.946340 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:16.946244 2550 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-14.ec2.internal\" not found" Jul 28 17:10:16.966916 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:16.966888 2550 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:10:17.020240 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.020207 2550 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:10:17.089530 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.089493 2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-14.ec2.internal" Jul 28 17:10:17.106904 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.106874 2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 17:10:17.107693 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.107678 2550 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-14.ec2.internal" Jul 28 17:10:17.119398 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.119375 2550 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 28 17:10:17.570413 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.570374 2550 apiserver.go:52] "Watching apiserver" Jul 28 17:10:17.582610 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.582576 2550 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 28 17:10:17.583904 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.583851 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-14.ec2.internal","openshift-dns/node-resolver-vpsrj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-14.ec2.internal","openshift-multus/multus-4428b","openshift-multus/network-metrics-daemon-4l6cw","openshift-network-diagnostics/network-check-target-kh2q2","openshift-network-operator/iptables-alerter-f5tgb","kube-system/global-pull-secret-syncer-gzjsf","kube-system/konnectivity-agent-g6hlp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp","openshift-cluster-node-tuning-operator/tuned-x45p5","openshift-image-registry/node-ca-jblj2","openshift-multus/multus-additional-cni-plugins-qslrz","openshift-ovn-kubernetes/ovnkube-node-4p62f"] Jul 28 17:10:17.586823 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.586799 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f5tgb" Jul 28 17:10:17.589232 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.589210 2550 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 28 17:10:17.589791 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.589766 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.590810 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.590672 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:10:17.590810 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.590680 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 28 17:10:17.590810 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.590750 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jz6bz\"" Jul 28 17:10:17.591026 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.590982 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 28 17:10:17.592580 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.592553 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.594135 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.594107 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tzw6b\"" Jul 28 17:10:17.594265 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.594248 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 28 17:10:17.594313 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.594290 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 28 17:10:17.594313 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.594298 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 28 17:10:17.594417 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.594115 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 28 17:10:17.594625 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.594607 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 28 17:10:17.594982 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.594966 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4428b" Jul 28 17:10:17.599237 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.596213 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 28 17:10:17.599237 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.596614 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6vjvp\"" Jul 28 17:10:17.599237 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.596710 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 28 17:10:17.599237 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.598011 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 28 17:10:17.599237 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.598230 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 28 17:10:17.599237 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.598483 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 28 17:10:17.599237 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.598734 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 28 17:10:17.599237 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.598878 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 28 17:10:17.599237 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.598981 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-r7w8h\"" Jul 28 17:10:17.599648 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.599256 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:17.599648 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:17.599364 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4l6cw" podUID="6cb7d969-3477-425b-9bf1-3192c9d7d94c" Jul 28 17:10:17.599648 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.599601 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d9ae751d-09fa-4166-b47b-0e069281d796-os-release\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.599810 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.599662 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-var-lib-openvswitch\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.599810 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.599697 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-host-run-ovn-kubernetes\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.599810 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.599724 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-host-cni-bin\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.600024 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.599802 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ed8e1716-5cda-4a93-b714-15d88545f8bd-ovnkube-config\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.600024 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.599845 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ed8e1716-5cda-4a93-b714-15d88545f8bd-env-overrides\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.600024 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.599878 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpqng\" (UniqueName: \"kubernetes.io/projected/491ac4f3-9300-4870-bf6e-d48f1c35a53f-kube-api-access-bpqng\") pod \"iptables-alerter-f5tgb\" (UID: \"491ac4f3-9300-4870-bf6e-d48f1c35a53f\") " pod="openshift-network-operator/iptables-alerter-f5tgb" Jul 28 17:10:17.600024 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.599945 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d9ae751d-09fa-4166-b47b-0e069281d796-cni-binary-copy\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.600024 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.599983 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/491ac4f3-9300-4870-bf6e-d48f1c35a53f-iptables-alerter-script\") pod \"iptables-alerter-f5tgb\" (UID: \"491ac4f3-9300-4870-bf6e-d48f1c35a53f\") " pod="openshift-network-operator/iptables-alerter-f5tgb" Jul 28 17:10:17.600289 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.600035 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d9ae751d-09fa-4166-b47b-0e069281d796-cnibin\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.600289 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.600088 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d9ae751d-09fa-4166-b47b-0e069281d796-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.600289 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.600148 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-host-slash\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.600672 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.600267 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-run-openvswitch\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.600750 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.600700 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-run-ovn\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.600750 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.600731 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-host-cni-netd\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.600856 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.600757 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ed8e1716-5cda-4a93-b714-15d88545f8bd-ovnkube-script-lib\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.600856 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.600786 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-host-kubelet\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.600856 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.600811 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-log-socket\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.600856 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.600836 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ed8e1716-5cda-4a93-b714-15d88545f8bd-ovn-node-metrics-cert\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.601050 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.600862 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d9ae751d-09fa-4166-b47b-0e069281d796-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.601050 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.600887 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d9ae751d-09fa-4166-b47b-0e069281d796-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.601050 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.600914 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdvcg\" (UniqueName: \"kubernetes.io/projected/d9ae751d-09fa-4166-b47b-0e069281d796-kube-api-access-jdvcg\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.601050 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.600938 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-etc-openvswitch\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.601050 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.600962 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-node-log\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.601050 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.600989 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.601050 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.601017 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-systemd-units\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.601050 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.601041 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-host-run-netns\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.601319 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.601080 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-run-systemd\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.601319 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.601106 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d9ae751d-09fa-4166-b47b-0e069281d796-system-cni-dir\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.601319 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.601131 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/491ac4f3-9300-4870-bf6e-d48f1c35a53f-host-slash\") pod \"iptables-alerter-f5tgb\" (UID: \"491ac4f3-9300-4870-bf6e-d48f1c35a53f\") " pod="openshift-network-operator/iptables-alerter-f5tgb" Jul 28 17:10:17.601319 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.601157 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hq24\" (UniqueName: \"kubernetes.io/projected/ed8e1716-5cda-4a93-b714-15d88545f8bd-kube-api-access-2hq24\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.602549 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.602526 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:17.602646 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:17.602615 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kh2q2" podUID="50d07fcb-69de-4546-a128-9549657dd856" Jul 28 17:10:17.605397 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.605172 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g6hlp" Jul 28 17:10:17.607600 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.607580 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:17.607690 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:17.607639 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gzjsf" podUID="143142e3-d09f-4dde-bf59-1fffafc3e2fc" Jul 28 17:10:17.610376 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.610179 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vpsrj" Jul 28 17:10:17.610779 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.610756 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 28 17:10:17.610901 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.610840 2550 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 28 17:10:17.610970 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.610933 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-wd6mf\"" Jul 28 17:10:17.611024 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.610971 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 28 17:10:17.613186 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.613057 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.615504 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.615295 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 28 17:10:17.615742 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.615628 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.616453 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.615965 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 28 17:10:17.616453 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.616218 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 28 17:10:17.616453 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.616279 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-prhxd\"" Jul 28 17:10:17.616737 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.616483 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 28 17:10:17.617618 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.617598 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 28 17:10:17.617966 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.617948 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wkzrz\"" Jul 28 17:10:17.618060 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.618042 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jblj2" Jul 28 17:10:17.618827 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.618792 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:10:17.619175 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.619131 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-xh9rt\"" Jul 28 17:10:17.619508 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.619483 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 28 17:10:17.633829 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.633377 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m5cqz\"" Jul 28 17:10:17.633829 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.633479 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 28 17:10:17.633829 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.633532 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 28 17:10:17.633829 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.633689 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 28 17:10:17.653643 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.653608 2550 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bkbdh" Jul 28 17:10:17.660463 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.660387 2550 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bkbdh" Jul 28 17:10:17.694574 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.694539 2550 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 28 17:10:17.701587 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.701508 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-etc-modprobe-d\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.701587 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.701559 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-etc-sysconfig\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.701587 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.701590 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-etc-kubernetes\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.701926 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.701616 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6mtk\" (UniqueName: \"kubernetes.io/projected/536d6825-43fe-4b71-b9d3-9d73177a2a56-kube-api-access-p6mtk\") pod \"aws-ebs-csi-driver-node-j2hkp\" (UID: \"536d6825-43fe-4b71-b9d3-9d73177a2a56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.701926 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.701642 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d9ae751d-09fa-4166-b47b-0e069281d796-system-cni-dir\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.701926 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.701657 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-etc-sysctl-conf\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.701926 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.701679 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-cnibin\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.701926 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.701695 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-cni-binary-copy\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.701926 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.701712 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-host-var-lib-cni-bin\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.701926 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.701730 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/491ac4f3-9300-4870-bf6e-d48f1c35a53f-host-slash\") pod \"iptables-alerter-f5tgb\" (UID: \"491ac4f3-9300-4870-bf6e-d48f1c35a53f\") " pod="openshift-network-operator/iptables-alerter-f5tgb" Jul 28 17:10:17.701926 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.701755 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hq24\" (UniqueName: \"kubernetes.io/projected/ed8e1716-5cda-4a93-b714-15d88545f8bd-kube-api-access-2hq24\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.701926 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.701775 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-run\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.701926 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.701791 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-multus-cni-dir\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.701926 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.701807 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-multus-socket-dir-parent\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.701926 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.701825 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-var-lib-openvswitch\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.701926 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.701846 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-host-cni-bin\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.701926 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.701871 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ed8e1716-5cda-4a93-b714-15d88545f8bd-ovnkube-config\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.701926 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.701897 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ed8e1716-5cda-4a93-b714-15d88545f8bd-env-overrides\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.701926 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.701923 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8cz5\" (UniqueName: \"kubernetes.io/projected/56e96be9-8647-407e-b2d9-f6718d894258-kube-api-access-v8cz5\") pod \"node-resolver-vpsrj\" (UID: \"56e96be9-8647-407e-b2d9-f6718d894258\") " pod="openshift-dns/node-resolver-vpsrj" Jul 28 17:10:17.702799 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.701949 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-etc-systemd\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.702799 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.701974 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-sys\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.702799 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702000 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d9ae751d-09fa-4166-b47b-0e069281d796-cni-binary-copy\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.702799 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702024 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-var-lib-kubelet\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.702799 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702051 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-host-run-multus-certs\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.702799 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702076 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/143142e3-d09f-4dde-bf59-1fffafc3e2fc-kubelet-config\") pod \"global-pull-secret-syncer-gzjsf\" (UID: \"143142e3-d09f-4dde-bf59-1fffafc3e2fc\") " pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:17.702799 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702101 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/536d6825-43fe-4b71-b9d3-9d73177a2a56-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j2hkp\" (UID: \"536d6825-43fe-4b71-b9d3-9d73177a2a56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.702799 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702125 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/536d6825-43fe-4b71-b9d3-9d73177a2a56-socket-dir\") pod \"aws-ebs-csi-driver-node-j2hkp\" (UID: \"536d6825-43fe-4b71-b9d3-9d73177a2a56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.702799 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702149 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/536d6825-43fe-4b71-b9d3-9d73177a2a56-device-dir\") pod \"aws-ebs-csi-driver-node-j2hkp\" (UID: \"536d6825-43fe-4b71-b9d3-9d73177a2a56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.702799 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702177 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d9ae751d-09fa-4166-b47b-0e069281d796-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.702799 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702203 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-host-slash\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.702799 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702237 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-run-ovn\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.702799 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702262 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-host-cni-netd\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.702799 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702287 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-system-cni-dir\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.702799 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702312 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/536d6825-43fe-4b71-b9d3-9d73177a2a56-registration-dir\") pod \"aws-ebs-csi-driver-node-j2hkp\" (UID: \"536d6825-43fe-4b71-b9d3-9d73177a2a56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.702799 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702337 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngl54\" (UniqueName: \"kubernetes.io/projected/c55b772a-3564-4bbe-b57d-073f7d731c17-kube-api-access-ngl54\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.703671 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702383 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj2zn\" (UniqueName: \"kubernetes.io/projected/c05afc7b-dee8-4de5-956e-336b79ac9867-kube-api-access-pj2zn\") pod \"node-ca-jblj2\" (UID: \"c05afc7b-dee8-4de5-956e-336b79ac9867\") " pod="openshift-image-registry/node-ca-jblj2" Jul 28 17:10:17.703671 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702412 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-host-kubelet\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.703671 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702437 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-host-run-k8s-cni-cncf-io\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.703671 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702466 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d9ae751d-09fa-4166-b47b-0e069281d796-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.703671 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702510 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.703671 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702538 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-multus-daemon-config\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.703671 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702564 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/143142e3-d09f-4dde-bf59-1fffafc3e2fc-dbus\") pod \"global-pull-secret-syncer-gzjsf\" (UID: \"143142e3-d09f-4dde-bf59-1fffafc3e2fc\") " pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:17.703671 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702595 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-systemd-units\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.703671 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702681 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-etc-sysctl-d\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.703671 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702706 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-os-release\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.703671 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702732 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-host-run-netns\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.703671 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702767 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bj4kl\" (UniqueName: \"kubernetes.io/projected/6cb7d969-3477-425b-9bf1-3192c9d7d94c-kube-api-access-bj4kl\") pod \"network-metrics-daemon-4l6cw\" (UID: \"6cb7d969-3477-425b-9bf1-3192c9d7d94c\") " pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:17.703671 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702794 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c05afc7b-dee8-4de5-956e-336b79ac9867-serviceca\") pod \"node-ca-jblj2\" (UID: \"c05afc7b-dee8-4de5-956e-336b79ac9867\") " pod="openshift-image-registry/node-ca-jblj2" Jul 28 17:10:17.703671 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702821 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-host-var-lib-kubelet\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.703671 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702847 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/536d6825-43fe-4b71-b9d3-9d73177a2a56-etc-selinux\") pod \"aws-ebs-csi-driver-node-j2hkp\" (UID: \"536d6825-43fe-4b71-b9d3-9d73177a2a56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.703671 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702872 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c55b772a-3564-4bbe-b57d-073f7d731c17-tmp\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.703671 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702898 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-host-var-lib-cni-multus\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.704515 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702927 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzr4x\" (UniqueName: \"kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x\") pod \"network-check-target-kh2q2\" (UID: \"50d07fcb-69de-4546-a128-9549657dd856\") " pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:17.704515 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702953 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs\") pod \"network-metrics-daemon-4l6cw\" (UID: \"6cb7d969-3477-425b-9bf1-3192c9d7d94c\") " pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:17.704515 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.702978 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c55b772a-3564-4bbe-b57d-073f7d731c17-etc-tuned\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.704515 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703002 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c05afc7b-dee8-4de5-956e-336b79ac9867-host\") pod \"node-ca-jblj2\" (UID: \"c05afc7b-dee8-4de5-956e-336b79ac9867\") " pod="openshift-image-registry/node-ca-jblj2" Jul 28 17:10:17.704515 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703018 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ed8e1716-5cda-4a93-b714-15d88545f8bd-ovnkube-config\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.704515 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703037 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d9ae751d-09fa-4166-b47b-0e069281d796-os-release\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.704515 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703066 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-host-run-ovn-kubernetes\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.704515 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703101 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-etc-kubernetes\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.704515 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703182 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-host\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.704515 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703197 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d9ae751d-09fa-4166-b47b-0e069281d796-system-cni-dir\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.704515 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703208 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/536d6825-43fe-4b71-b9d3-9d73177a2a56-sys-fs\") pod \"aws-ebs-csi-driver-node-j2hkp\" (UID: \"536d6825-43fe-4b71-b9d3-9d73177a2a56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.704515 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703263 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpqng\" (UniqueName: \"kubernetes.io/projected/491ac4f3-9300-4870-bf6e-d48f1c35a53f-kube-api-access-bpqng\") pod \"iptables-alerter-f5tgb\" (UID: \"491ac4f3-9300-4870-bf6e-d48f1c35a53f\") " pod="openshift-network-operator/iptables-alerter-f5tgb" Jul 28 17:10:17.704515 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703296 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6457cb80-cd0c-4342-8fab-4ad86a56af24-konnectivity-ca\") pod \"konnectivity-agent-g6hlp\" (UID: \"6457cb80-cd0c-4342-8fab-4ad86a56af24\") " pod="kube-system/konnectivity-agent-g6hlp" Jul 28 17:10:17.704515 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703328 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/56e96be9-8647-407e-b2d9-f6718d894258-hosts-file\") pod \"node-resolver-vpsrj\" (UID: \"56e96be9-8647-407e-b2d9-f6718d894258\") " pod="openshift-dns/node-resolver-vpsrj" Jul 28 17:10:17.704515 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703355 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-multus-conf-dir\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.704515 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703382 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/491ac4f3-9300-4870-bf6e-d48f1c35a53f-iptables-alerter-script\") pod \"iptables-alerter-f5tgb\" (UID: \"491ac4f3-9300-4870-bf6e-d48f1c35a53f\") " pod="openshift-network-operator/iptables-alerter-f5tgb" Jul 28 17:10:17.704515 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703411 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d9ae751d-09fa-4166-b47b-0e069281d796-cnibin\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.705439 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703439 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-run-openvswitch\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.705439 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703487 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ed8e1716-5cda-4a93-b714-15d88545f8bd-ovnkube-script-lib\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.705439 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703517 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret\") pod \"global-pull-secret-syncer-gzjsf\" (UID: \"143142e3-d09f-4dde-bf59-1fffafc3e2fc\") " pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:17.705439 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703567 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-log-socket\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.705439 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703593 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ed8e1716-5cda-4a93-b714-15d88545f8bd-ovn-node-metrics-cert\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.705439 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703621 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6457cb80-cd0c-4342-8fab-4ad86a56af24-agent-certs\") pod \"konnectivity-agent-g6hlp\" (UID: \"6457cb80-cd0c-4342-8fab-4ad86a56af24\") " pod="kube-system/konnectivity-agent-g6hlp" Jul 28 17:10:17.705439 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703643 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-lib-modules\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.705439 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703655 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ed8e1716-5cda-4a93-b714-15d88545f8bd-env-overrides\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.705439 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703667 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-hostroot\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.705439 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703702 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl2sv\" (UniqueName: \"kubernetes.io/projected/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-kube-api-access-kl2sv\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.705439 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703725 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d9ae751d-09fa-4166-b47b-0e069281d796-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.705439 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703921 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.705439 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703931 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d9ae751d-09fa-4166-b47b-0e069281d796-os-release\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.705439 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.703991 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdvcg\" (UniqueName: \"kubernetes.io/projected/d9ae751d-09fa-4166-b47b-0e069281d796-kube-api-access-jdvcg\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.705439 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.704007 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-host-run-ovn-kubernetes\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.705439 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.704018 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-etc-openvswitch\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.705439 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.704058 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-node-log\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.706309 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.704070 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-etc-openvswitch\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.706309 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.704095 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56e96be9-8647-407e-b2d9-f6718d894258-tmp-dir\") pod \"node-resolver-vpsrj\" (UID: \"56e96be9-8647-407e-b2d9-f6718d894258\") " pod="openshift-dns/node-resolver-vpsrj" Jul 28 17:10:17.706309 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.704128 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-host-run-netns\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.706309 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.704157 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-run-systemd\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.706309 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.704234 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-run-systemd\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.706309 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.704442 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d9ae751d-09fa-4166-b47b-0e069281d796-cni-binary-copy\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.706309 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.704516 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-host-kubelet\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.706309 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.704554 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d9ae751d-09fa-4166-b47b-0e069281d796-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.706309 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.704626 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/491ac4f3-9300-4870-bf6e-d48f1c35a53f-host-slash\") pod \"iptables-alerter-f5tgb\" (UID: \"491ac4f3-9300-4870-bf6e-d48f1c35a53f\") " pod="openshift-network-operator/iptables-alerter-f5tgb" Jul 28 17:10:17.706309 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.704845 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/491ac4f3-9300-4870-bf6e-d48f1c35a53f-iptables-alerter-script\") pod \"iptables-alerter-f5tgb\" (UID: \"491ac4f3-9300-4870-bf6e-d48f1c35a53f\") " pod="openshift-network-operator/iptables-alerter-f5tgb" Jul 28 17:10:17.706309 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.704903 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d9ae751d-09fa-4166-b47b-0e069281d796-cnibin\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.706309 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.704944 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-run-openvswitch\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.706309 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.704986 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-node-log\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.706309 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.704989 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d9ae751d-09fa-4166-b47b-0e069281d796-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.706309 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.705016 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-host-slash\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.706309 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.705041 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-log-socket\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.706309 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.705054 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-host-cni-bin\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.707274 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.705062 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-systemd-units\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.707274 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.705093 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-var-lib-openvswitch\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.707274 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.705137 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-run-ovn\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.707274 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.705145 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d9ae751d-09fa-4166-b47b-0e069281d796-tuning-conf-dir\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.707274 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.705176 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-host-run-netns\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.707274 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.704987 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ed8e1716-5cda-4a93-b714-15d88545f8bd-ovnkube-script-lib\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.707274 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.705298 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ed8e1716-5cda-4a93-b714-15d88545f8bd-host-cni-netd\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.707274 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.705339 2550 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 28 17:10:17.709634 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.709608 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ed8e1716-5cda-4a93-b714-15d88545f8bd-ovn-node-metrics-cert\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.734051 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.734013 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpqng\" (UniqueName: \"kubernetes.io/projected/491ac4f3-9300-4870-bf6e-d48f1c35a53f-kube-api-access-bpqng\") pod \"iptables-alerter-f5tgb\" (UID: \"491ac4f3-9300-4870-bf6e-d48f1c35a53f\") " pod="openshift-network-operator/iptables-alerter-f5tgb" Jul 28 17:10:17.734228 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.734208 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdvcg\" (UniqueName: \"kubernetes.io/projected/d9ae751d-09fa-4166-b47b-0e069281d796-kube-api-access-jdvcg\") pod \"multus-additional-cni-plugins-qslrz\" (UID: \"d9ae751d-09fa-4166-b47b-0e069281d796\") " pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.734290 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.734210 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hq24\" (UniqueName: \"kubernetes.io/projected/ed8e1716-5cda-4a93-b714-15d88545f8bd-kube-api-access-2hq24\") pod \"ovnkube-node-4p62f\" (UID: \"ed8e1716-5cda-4a93-b714-15d88545f8bd\") " pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.742996 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.742964 2550 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:10:17.804638 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.804599 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c05afc7b-dee8-4de5-956e-336b79ac9867-serviceca\") pod \"node-ca-jblj2\" (UID: \"c05afc7b-dee8-4de5-956e-336b79ac9867\") " pod="openshift-image-registry/node-ca-jblj2" Jul 28 17:10:17.804838 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.804742 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-host-var-lib-kubelet\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.804838 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.804785 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/536d6825-43fe-4b71-b9d3-9d73177a2a56-etc-selinux\") pod \"aws-ebs-csi-driver-node-j2hkp\" (UID: \"536d6825-43fe-4b71-b9d3-9d73177a2a56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.804838 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.804813 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c55b772a-3564-4bbe-b57d-073f7d731c17-tmp\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.805020 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.804838 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-host-var-lib-cni-multus\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.805020 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.804879 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzr4x\" (UniqueName: \"kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x\") pod \"network-check-target-kh2q2\" (UID: \"50d07fcb-69de-4546-a128-9549657dd856\") " pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:17.805020 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.804909 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs\") pod \"network-metrics-daemon-4l6cw\" (UID: \"6cb7d969-3477-425b-9bf1-3192c9d7d94c\") " pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:17.805020 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.804933 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c55b772a-3564-4bbe-b57d-073f7d731c17-etc-tuned\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.805020 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805003 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-host-var-lib-kubelet\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.805274 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805060 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c05afc7b-dee8-4de5-956e-336b79ac9867-host\") pod \"node-ca-jblj2\" (UID: \"c05afc7b-dee8-4de5-956e-336b79ac9867\") " pod="openshift-image-registry/node-ca-jblj2" Jul 28 17:10:17.805274 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805098 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-etc-kubernetes\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.805274 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805172 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-host\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.805274 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805190 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c05afc7b-dee8-4de5-956e-336b79ac9867-serviceca\") pod \"node-ca-jblj2\" (UID: \"c05afc7b-dee8-4de5-956e-336b79ac9867\") " pod="openshift-image-registry/node-ca-jblj2" Jul 28 17:10:17.805274 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805201 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/536d6825-43fe-4b71-b9d3-9d73177a2a56-sys-fs\") pod \"aws-ebs-csi-driver-node-j2hkp\" (UID: \"536d6825-43fe-4b71-b9d3-9d73177a2a56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.805274 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805217 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-host-var-lib-cni-multus\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.805274 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805261 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/536d6825-43fe-4b71-b9d3-9d73177a2a56-etc-selinux\") pod \"aws-ebs-csi-driver-node-j2hkp\" (UID: \"536d6825-43fe-4b71-b9d3-9d73177a2a56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.805664 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805265 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6457cb80-cd0c-4342-8fab-4ad86a56af24-konnectivity-ca\") pod \"konnectivity-agent-g6hlp\" (UID: \"6457cb80-cd0c-4342-8fab-4ad86a56af24\") " pod="kube-system/konnectivity-agent-g6hlp" Jul 28 17:10:17.805664 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805293 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-etc-kubernetes\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.805664 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805297 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-host\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.805664 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805380 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/536d6825-43fe-4b71-b9d3-9d73177a2a56-sys-fs\") pod \"aws-ebs-csi-driver-node-j2hkp\" (UID: \"536d6825-43fe-4b71-b9d3-9d73177a2a56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.805664 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805512 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/56e96be9-8647-407e-b2d9-f6718d894258-hosts-file\") pod \"node-resolver-vpsrj\" (UID: \"56e96be9-8647-407e-b2d9-f6718d894258\") " pod="openshift-dns/node-resolver-vpsrj" Jul 28 17:10:17.805664 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805585 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-multus-conf-dir\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.805664 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805588 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c05afc7b-dee8-4de5-956e-336b79ac9867-host\") pod \"node-ca-jblj2\" (UID: \"c05afc7b-dee8-4de5-956e-336b79ac9867\") " pod="openshift-image-registry/node-ca-jblj2" Jul 28 17:10:17.805664 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805621 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret\") pod \"global-pull-secret-syncer-gzjsf\" (UID: \"143142e3-d09f-4dde-bf59-1fffafc3e2fc\") " pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:17.805664 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805652 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6457cb80-cd0c-4342-8fab-4ad86a56af24-agent-certs\") pod \"konnectivity-agent-g6hlp\" (UID: \"6457cb80-cd0c-4342-8fab-4ad86a56af24\") " pod="kube-system/konnectivity-agent-g6hlp" Jul 28 17:10:17.806183 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805679 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-lib-modules\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.806183 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:17.805714 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:17.806183 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805771 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-hostroot\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.806183 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:17.805806 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs podName:6cb7d969-3477-425b-9bf1-3192c9d7d94c nodeName:}" failed. No retries permitted until 2026-07-28 17:10:18.305782205 +0000 UTC m=+3.034283537 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs") pod "network-metrics-daemon-4l6cw" (UID: "6cb7d969-3477-425b-9bf1-3192c9d7d94c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:17.806183 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805715 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-hostroot\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.806183 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805828 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/56e96be9-8647-407e-b2d9-f6718d894258-hosts-file\") pod \"node-resolver-vpsrj\" (UID: \"56e96be9-8647-407e-b2d9-f6718d894258\") " pod="openshift-dns/node-resolver-vpsrj" Jul 28 17:10:17.806183 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805973 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6457cb80-cd0c-4342-8fab-4ad86a56af24-konnectivity-ca\") pod \"konnectivity-agent-g6hlp\" (UID: \"6457cb80-cd0c-4342-8fab-4ad86a56af24\") " pod="kube-system/konnectivity-agent-g6hlp" Jul 28 17:10:17.806183 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806037 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-multus-conf-dir\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.806183 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.805848 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl2sv\" (UniqueName: \"kubernetes.io/projected/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-kube-api-access-kl2sv\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.806183 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806173 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-lib-modules\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.806718 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806206 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56e96be9-8647-407e-b2d9-f6718d894258-tmp-dir\") pod \"node-resolver-vpsrj\" (UID: \"56e96be9-8647-407e-b2d9-f6718d894258\") " pod="openshift-dns/node-resolver-vpsrj" Jul 28 17:10:17.806718 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806237 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-etc-modprobe-d\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.806718 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:17.806253 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:17.806718 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806261 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-etc-sysconfig\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.806718 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806285 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-etc-kubernetes\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.806718 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:17.806302 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret podName:143142e3-d09f-4dde-bf59-1fffafc3e2fc nodeName:}" failed. No retries permitted until 2026-07-28 17:10:18.306286834 +0000 UTC m=+3.034788153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret") pod "global-pull-secret-syncer-gzjsf" (UID: "143142e3-d09f-4dde-bf59-1fffafc3e2fc") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:17.806718 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806329 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-etc-kubernetes\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.806718 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806335 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6mtk\" (UniqueName: \"kubernetes.io/projected/536d6825-43fe-4b71-b9d3-9d73177a2a56-kube-api-access-p6mtk\") pod \"aws-ebs-csi-driver-node-j2hkp\" (UID: \"536d6825-43fe-4b71-b9d3-9d73177a2a56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.806718 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806388 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-etc-sysctl-conf\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.806718 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806422 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-cnibin\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.806718 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806436 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-etc-modprobe-d\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.806718 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806447 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-cni-binary-copy\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.806718 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806507 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-host-var-lib-cni-bin\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.806718 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806511 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-etc-sysconfig\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.806718 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806535 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-run\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.806718 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806545 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/56e96be9-8647-407e-b2d9-f6718d894258-tmp-dir\") pod \"node-resolver-vpsrj\" (UID: \"56e96be9-8647-407e-b2d9-f6718d894258\") " pod="openshift-dns/node-resolver-vpsrj" Jul 28 17:10:17.806718 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806559 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-multus-cni-dir\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.806718 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806586 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-multus-socket-dir-parent\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.807616 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806640 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-etc-sysctl-conf\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.807616 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806645 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-host-var-lib-cni-bin\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.807616 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806641 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8cz5\" (UniqueName: \"kubernetes.io/projected/56e96be9-8647-407e-b2d9-f6718d894258-kube-api-access-v8cz5\") pod \"node-resolver-vpsrj\" (UID: \"56e96be9-8647-407e-b2d9-f6718d894258\") " pod="openshift-dns/node-resolver-vpsrj" Jul 28 17:10:17.807616 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806684 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-etc-systemd\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.807616 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806700 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-cnibin\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.807616 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806709 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-sys\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.807616 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806737 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-var-lib-kubelet\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.807616 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806768 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-host-run-multus-certs\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.807616 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806799 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/143142e3-d09f-4dde-bf59-1fffafc3e2fc-kubelet-config\") pod \"global-pull-secret-syncer-gzjsf\" (UID: \"143142e3-d09f-4dde-bf59-1fffafc3e2fc\") " pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:17.807616 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806825 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/536d6825-43fe-4b71-b9d3-9d73177a2a56-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j2hkp\" (UID: \"536d6825-43fe-4b71-b9d3-9d73177a2a56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.807616 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806826 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-multus-socket-dir-parent\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.807616 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806855 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/536d6825-43fe-4b71-b9d3-9d73177a2a56-socket-dir\") pod \"aws-ebs-csi-driver-node-j2hkp\" (UID: \"536d6825-43fe-4b71-b9d3-9d73177a2a56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.807616 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806897 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/536d6825-43fe-4b71-b9d3-9d73177a2a56-device-dir\") pod \"aws-ebs-csi-driver-node-j2hkp\" (UID: \"536d6825-43fe-4b71-b9d3-9d73177a2a56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.807616 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806930 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-system-cni-dir\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.807616 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806958 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/536d6825-43fe-4b71-b9d3-9d73177a2a56-registration-dir\") pod \"aws-ebs-csi-driver-node-j2hkp\" (UID: \"536d6825-43fe-4b71-b9d3-9d73177a2a56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.807616 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806986 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngl54\" (UniqueName: \"kubernetes.io/projected/c55b772a-3564-4bbe-b57d-073f7d731c17-kube-api-access-ngl54\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.807616 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806992 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/536d6825-43fe-4b71-b9d3-9d73177a2a56-socket-dir\") pod \"aws-ebs-csi-driver-node-j2hkp\" (UID: \"536d6825-43fe-4b71-b9d3-9d73177a2a56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.808522 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807023 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj2zn\" (UniqueName: \"kubernetes.io/projected/c05afc7b-dee8-4de5-956e-336b79ac9867-kube-api-access-pj2zn\") pod \"node-ca-jblj2\" (UID: \"c05afc7b-dee8-4de5-956e-336b79ac9867\") " pod="openshift-image-registry/node-ca-jblj2" Jul 28 17:10:17.808522 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807063 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-host-run-k8s-cni-cncf-io\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.808522 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807118 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-multus-daemon-config\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.808522 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807145 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/143142e3-d09f-4dde-bf59-1fffafc3e2fc-dbus\") pod \"global-pull-secret-syncer-gzjsf\" (UID: \"143142e3-d09f-4dde-bf59-1fffafc3e2fc\") " pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:17.808522 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807173 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-etc-sysctl-d\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.808522 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807210 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-os-release\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.808522 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807234 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-host-run-netns\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.808522 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807262 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bj4kl\" (UniqueName: \"kubernetes.io/projected/6cb7d969-3477-425b-9bf1-3192c9d7d94c-kube-api-access-bj4kl\") pod \"network-metrics-daemon-4l6cw\" (UID: \"6cb7d969-3477-425b-9bf1-3192c9d7d94c\") " pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:17.808522 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807310 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-etc-systemd\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.808522 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807369 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-sys\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.808522 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807425 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-var-lib-kubelet\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.808522 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807490 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-host-run-multus-certs\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.808522 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.806772 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-run\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.808522 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807515 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-cni-binary-copy\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.808522 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807600 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/536d6825-43fe-4b71-b9d3-9d73177a2a56-kubelet-dir\") pod \"aws-ebs-csi-driver-node-j2hkp\" (UID: \"536d6825-43fe-4b71-b9d3-9d73177a2a56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.808522 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807599 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/143142e3-d09f-4dde-bf59-1fffafc3e2fc-kubelet-config\") pod \"global-pull-secret-syncer-gzjsf\" (UID: \"143142e3-d09f-4dde-bf59-1fffafc3e2fc\") " pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:17.808522 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807633 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-multus-cni-dir\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.808522 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807642 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-host-run-k8s-cni-cncf-io\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.809387 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807670 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/536d6825-43fe-4b71-b9d3-9d73177a2a56-registration-dir\") pod \"aws-ebs-csi-driver-node-j2hkp\" (UID: \"536d6825-43fe-4b71-b9d3-9d73177a2a56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.809387 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807689 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/536d6825-43fe-4b71-b9d3-9d73177a2a56-device-dir\") pod \"aws-ebs-csi-driver-node-j2hkp\" (UID: \"536d6825-43fe-4b71-b9d3-9d73177a2a56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.809387 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807727 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-system-cni-dir\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.809387 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807824 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c55b772a-3564-4bbe-b57d-073f7d731c17-etc-sysctl-d\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.809387 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807879 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-host-run-netns\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.809387 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807912 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-os-release\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.809387 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.807985 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/143142e3-d09f-4dde-bf59-1fffafc3e2fc-dbus\") pod \"global-pull-secret-syncer-gzjsf\" (UID: \"143142e3-d09f-4dde-bf59-1fffafc3e2fc\") " pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:17.809387 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.808254 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-multus-daemon-config\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.809387 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.808253 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c55b772a-3564-4bbe-b57d-073f7d731c17-tmp\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.809387 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.808537 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c55b772a-3564-4bbe-b57d-073f7d731c17-etc-tuned\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.809387 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.808960 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6457cb80-cd0c-4342-8fab-4ad86a56af24-agent-certs\") pod \"konnectivity-agent-g6hlp\" (UID: \"6457cb80-cd0c-4342-8fab-4ad86a56af24\") " pod="kube-system/konnectivity-agent-g6hlp" Jul 28 17:10:17.813150 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:17.813124 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:10:17.813276 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:17.813159 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:10:17.813276 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:17.813176 2550 projected.go:196] Error preparing data for projected volume kube-api-access-tzr4x for pod openshift-network-diagnostics/network-check-target-kh2q2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:17.813276 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:17.813250 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x podName:50d07fcb-69de-4546-a128-9549657dd856 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:18.313232069 +0000 UTC m=+3.041733406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tzr4x" (UniqueName: "kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x") pod "network-check-target-kh2q2" (UID: "50d07fcb-69de-4546-a128-9549657dd856") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:17.825698 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.825621 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8cz5\" (UniqueName: \"kubernetes.io/projected/56e96be9-8647-407e-b2d9-f6718d894258-kube-api-access-v8cz5\") pod \"node-resolver-vpsrj\" (UID: \"56e96be9-8647-407e-b2d9-f6718d894258\") " pod="openshift-dns/node-resolver-vpsrj" Jul 28 17:10:17.826569 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.826531 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6mtk\" (UniqueName: \"kubernetes.io/projected/536d6825-43fe-4b71-b9d3-9d73177a2a56-kube-api-access-p6mtk\") pod \"aws-ebs-csi-driver-node-j2hkp\" (UID: \"536d6825-43fe-4b71-b9d3-9d73177a2a56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.826831 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.826805 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bj4kl\" (UniqueName: \"kubernetes.io/projected/6cb7d969-3477-425b-9bf1-3192c9d7d94c-kube-api-access-bj4kl\") pod \"network-metrics-daemon-4l6cw\" (UID: \"6cb7d969-3477-425b-9bf1-3192c9d7d94c\") " pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:17.826919 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.826865 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngl54\" (UniqueName: \"kubernetes.io/projected/c55b772a-3564-4bbe-b57d-073f7d731c17-kube-api-access-ngl54\") pod \"tuned-x45p5\" (UID: \"c55b772a-3564-4bbe-b57d-073f7d731c17\") " pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.827627 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.827454 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl2sv\" (UniqueName: \"kubernetes.io/projected/1dca65a6-b651-46fd-a9ef-fc4eb8ff667e-kube-api-access-kl2sv\") pod \"multus-4428b\" (UID: \"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e\") " pod="openshift-multus/multus-4428b" Jul 28 17:10:17.830598 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.830577 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj2zn\" (UniqueName: \"kubernetes.io/projected/c05afc7b-dee8-4de5-956e-336b79ac9867-kube-api-access-pj2zn\") pod \"node-ca-jblj2\" (UID: \"c05afc7b-dee8-4de5-956e-336b79ac9867\") " pod="openshift-image-registry/node-ca-jblj2" Jul 28 17:10:17.903431 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.903336 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f5tgb" Jul 28 17:10:17.912120 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.912085 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-qslrz" Jul 28 17:10:17.922603 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.922573 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:17.928705 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.928679 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4428b" Jul 28 17:10:17.936822 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.936777 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-g6hlp" Jul 28 17:10:17.944785 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.944759 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vpsrj" Jul 28 17:10:17.952823 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.952788 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" Jul 28 17:10:17.960752 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.960725 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-x45p5" Jul 28 17:10:17.966689 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:17.966667 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jblj2" Jul 28 17:10:18.310686 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:18.310643 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs\") pod \"network-metrics-daemon-4l6cw\" (UID: \"6cb7d969-3477-425b-9bf1-3192c9d7d94c\") " pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:18.310901 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:18.310804 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:18.310901 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:18.310811 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret\") pod \"global-pull-secret-syncer-gzjsf\" (UID: \"143142e3-d09f-4dde-bf59-1fffafc3e2fc\") " pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:18.310901 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:18.310876 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs podName:6cb7d969-3477-425b-9bf1-3192c9d7d94c nodeName:}" failed. No retries permitted until 2026-07-28 17:10:19.310855099 +0000 UTC m=+4.039356418 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs") pod "network-metrics-daemon-4l6cw" (UID: "6cb7d969-3477-425b-9bf1-3192c9d7d94c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:18.311080 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:18.310936 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:18.311080 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:18.310995 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret podName:143142e3-d09f-4dde-bf59-1fffafc3e2fc nodeName:}" failed. No retries permitted until 2026-07-28 17:10:19.310976565 +0000 UTC m=+4.039477890 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret") pod "global-pull-secret-syncer-gzjsf" (UID: "143142e3-d09f-4dde-bf59-1fffafc3e2fc") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:18.411878 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:18.411831 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzr4x\" (UniqueName: \"kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x\") pod \"network-check-target-kh2q2\" (UID: \"50d07fcb-69de-4546-a128-9549657dd856\") " pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:18.412052 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:18.412029 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:10:18.412100 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:18.412059 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:10:18.412100 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:18.412074 2550 projected.go:196] Error preparing data for projected volume kube-api-access-tzr4x for pod openshift-network-diagnostics/network-check-target-kh2q2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:18.412179 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:18.412139 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x podName:50d07fcb-69de-4546-a128-9549657dd856 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:19.41212195 +0000 UTC m=+4.140623265 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzr4x" (UniqueName: "kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x") pod "network-check-target-kh2q2" (UID: "50d07fcb-69de-4546-a128-9549657dd856") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:18.664092 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:18.663961 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:05:18 +0000 UTC" deadline="2028-02-12 04:17:20.035972051 +0000 UTC" Jul 28 17:10:18.664092 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:18.664029 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13523h7m1.371948124s" Jul 28 17:10:18.774501 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:18.774447 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc55b772a_3564_4bbe_b57d_073f7d731c17.slice/crio-b53037909718cafc81205d435a24decc0c1ce4b09bcfb7454bc4d3c68327c557 WatchSource:0}: Error finding container b53037909718cafc81205d435a24decc0c1ce4b09bcfb7454bc4d3c68327c557: Status 404 returned error can't find the container with id b53037909718cafc81205d435a24decc0c1ce4b09bcfb7454bc4d3c68327c557 Jul 28 17:10:18.775802 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:18.775772 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc05afc7b_dee8_4de5_956e_336b79ac9867.slice/crio-1b5bfc6c8848db25f229889fe2fa7c4d6b737fd866f82b5d93bc0cbe71bda904 WatchSource:0}: Error finding container 1b5bfc6c8848db25f229889fe2fa7c4d6b737fd866f82b5d93bc0cbe71bda904: Status 404 returned error can't find the container with id 1b5bfc6c8848db25f229889fe2fa7c4d6b737fd866f82b5d93bc0cbe71bda904 Jul 28 17:10:18.778417 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:18.778374 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dca65a6_b651_46fd_a9ef_fc4eb8ff667e.slice/crio-a421c557da20d1a34046b46e84b22a3d3fa763c2cc884cf629a2f8ae64181854 WatchSource:0}: Error finding container a421c557da20d1a34046b46e84b22a3d3fa763c2cc884cf629a2f8ae64181854: Status 404 returned error can't find the container with id a421c557da20d1a34046b46e84b22a3d3fa763c2cc884cf629a2f8ae64181854 Jul 28 17:10:18.780375 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:18.780343 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jblj2" event={"ID":"c05afc7b-dee8-4de5-956e-336b79ac9867","Type":"ContainerStarted","Data":"1b5bfc6c8848db25f229889fe2fa7c4d6b737fd866f82b5d93bc0cbe71bda904"} Jul 28 17:10:18.781910 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:18.781871 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod536d6825_43fe_4b71_b9d3_9d73177a2a56.slice/crio-e21bd7a8f6a31a2dfbd9ac4772bcb1e590d32c05615e8334288644be84aa564c WatchSource:0}: Error finding container e21bd7a8f6a31a2dfbd9ac4772bcb1e590d32c05615e8334288644be84aa564c: Status 404 returned error can't find the container with id e21bd7a8f6a31a2dfbd9ac4772bcb1e590d32c05615e8334288644be84aa564c Jul 28 17:10:18.782374 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:18.782253 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6457cb80_cd0c_4342_8fab_4ad86a56af24.slice/crio-528353ade5e379d0904cc528b48a75f82f02b9ef99985fa3370d4a4025f1900f WatchSource:0}: Error finding container 528353ade5e379d0904cc528b48a75f82f02b9ef99985fa3370d4a4025f1900f: Status 404 returned error can't find the container with id 528353ade5e379d0904cc528b48a75f82f02b9ef99985fa3370d4a4025f1900f Jul 28 17:10:18.782479 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:18.782450 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x45p5" event={"ID":"c55b772a-3564-4bbe-b57d-073f7d731c17","Type":"ContainerStarted","Data":"b53037909718cafc81205d435a24decc0c1ce4b09bcfb7454bc4d3c68327c557"} Jul 28 17:10:18.783231 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:18.783203 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56e96be9_8647_407e_b2d9_f6718d894258.slice/crio-2720c77c8b55c397818ee66a96355cca52b9cd888e8dd0d02b0769601bf50ef3 WatchSource:0}: Error finding container 2720c77c8b55c397818ee66a96355cca52b9cd888e8dd0d02b0769601bf50ef3: Status 404 returned error can't find the container with id 2720c77c8b55c397818ee66a96355cca52b9cd888e8dd0d02b0769601bf50ef3 Jul 28 17:10:18.784452 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:18.784431 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod491ac4f3_9300_4870_bf6e_d48f1c35a53f.slice/crio-3af94de6699f4ab92deaaa2b33f40e89f57fd82b2b8ad429cb3c5389c886d149 WatchSource:0}: Error finding container 3af94de6699f4ab92deaaa2b33f40e89f57fd82b2b8ad429cb3c5389c886d149: Status 404 returned error can't find the container with id 3af94de6699f4ab92deaaa2b33f40e89f57fd82b2b8ad429cb3c5389c886d149 Jul 28 17:10:18.786481 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:18.786181 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9ae751d_09fa_4166_b47b_0e069281d796.slice/crio-e4248fe57201f451d6837b05fd5956424256e530381db60d251f2d9a2a84bf1d WatchSource:0}: Error finding container e4248fe57201f451d6837b05fd5956424256e530381db60d251f2d9a2a84bf1d: Status 404 returned error can't find the container with id e4248fe57201f451d6837b05fd5956424256e530381db60d251f2d9a2a84bf1d Jul 28 17:10:18.786960 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:18.786939 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded8e1716_5cda_4a93_b714_15d88545f8bd.slice/crio-7118e9f79910db11602550e57a0742e177145c5b82300c6d9ce35deab62b3d9c WatchSource:0}: Error finding container 7118e9f79910db11602550e57a0742e177145c5b82300c6d9ce35deab62b3d9c: Status 404 returned error can't find the container with id 7118e9f79910db11602550e57a0742e177145c5b82300c6d9ce35deab62b3d9c Jul 28 17:10:18.951466 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:18.951228 2550 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 28 17:10:19.318518 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:19.318295 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs\") pod \"network-metrics-daemon-4l6cw\" (UID: \"6cb7d969-3477-425b-9bf1-3192c9d7d94c\") " pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:19.318518 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:19.318361 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret\") pod \"global-pull-secret-syncer-gzjsf\" (UID: \"143142e3-d09f-4dde-bf59-1fffafc3e2fc\") " pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:19.318812 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:19.318658 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:19.318812 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:19.318670 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:19.318812 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:19.318736 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs podName:6cb7d969-3477-425b-9bf1-3192c9d7d94c nodeName:}" failed. No retries permitted until 2026-07-28 17:10:21.318715331 +0000 UTC m=+6.047216651 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs") pod "network-metrics-daemon-4l6cw" (UID: "6cb7d969-3477-425b-9bf1-3192c9d7d94c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:19.318812 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:19.318760 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret podName:143142e3-d09f-4dde-bf59-1fffafc3e2fc nodeName:}" failed. No retries permitted until 2026-07-28 17:10:21.318747202 +0000 UTC m=+6.047248526 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret") pod "global-pull-secret-syncer-gzjsf" (UID: "143142e3-d09f-4dde-bf59-1fffafc3e2fc") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:19.423801 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:19.423749 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzr4x\" (UniqueName: \"kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x\") pod \"network-check-target-kh2q2\" (UID: \"50d07fcb-69de-4546-a128-9549657dd856\") " pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:19.424069 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:19.423928 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:10:19.424069 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:19.424050 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:10:19.424069 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:19.424067 2550 projected.go:196] Error preparing data for projected volume kube-api-access-tzr4x for pod openshift-network-diagnostics/network-check-target-kh2q2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:19.424249 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:19.424147 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x podName:50d07fcb-69de-4546-a128-9549657dd856 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:21.424125999 +0000 UTC m=+6.152627339 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzr4x" (UniqueName: "kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x") pod "network-check-target-kh2q2" (UID: "50d07fcb-69de-4546-a128-9549657dd856") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:19.664777 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:19.664651 2550 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 17:05:18 +0000 UTC" deadline="2028-02-19 22:15:59.055244642 +0000 UTC" Jul 28 17:10:19.664777 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:19.664723 2550 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13709h5m39.390525993s" Jul 28 17:10:19.770311 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:19.769308 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:19.770311 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:19.769530 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kh2q2" podUID="50d07fcb-69de-4546-a128-9549657dd856" Jul 28 17:10:19.770311 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:19.769993 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:19.770311 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:19.770094 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gzjsf" podUID="143142e3-d09f-4dde-bf59-1fffafc3e2fc" Jul 28 17:10:19.770311 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:19.770164 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:19.770311 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:19.770251 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4l6cw" podUID="6cb7d969-3477-425b-9bf1-3192c9d7d94c" Jul 28 17:10:19.804054 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:19.804009 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-14.ec2.internal" event={"ID":"7cc46c224c17195fb5410e94fda8e4a3","Type":"ContainerStarted","Data":"5c3a4efe1fe67ab1903f8e1574042a6150f05d5113b3c3cb978d55dcae6f030b"} Jul 28 17:10:19.817391 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:19.817351 2550 generic.go:356] "Generic (PLEG): container finished" podID="1b52734c50b11119e9322d7d16760bf3" containerID="0cb1363fe929ed16ea71a353ff463b77ab58a219ef0b93711c631b265b8b5754" exitCode=0 Jul 28 17:10:19.817649 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:19.817478 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-14.ec2.internal" event={"ID":"1b52734c50b11119e9322d7d16760bf3","Type":"ContainerDied","Data":"0cb1363fe929ed16ea71a353ff463b77ab58a219ef0b93711c631b265b8b5754"} Jul 28 17:10:19.822678 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:19.822531 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f5tgb" event={"ID":"491ac4f3-9300-4870-bf6e-d48f1c35a53f","Type":"ContainerStarted","Data":"3af94de6699f4ab92deaaa2b33f40e89f57fd82b2b8ad429cb3c5389c886d149"} Jul 28 17:10:19.825006 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:19.824793 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-14.ec2.internal" podStartSLOduration=2.824773143 podStartE2EDuration="2.824773143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:19.823795833 +0000 UTC m=+4.552297175" watchObservedRunningTime="2026-07-28 17:10:19.824773143 +0000 UTC m=+4.553274484" Jul 28 17:10:19.831896 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:19.831802 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4428b" event={"ID":"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e","Type":"ContainerStarted","Data":"a421c557da20d1a34046b46e84b22a3d3fa763c2cc884cf629a2f8ae64181854"} Jul 28 17:10:19.840183 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:19.840151 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g6hlp" event={"ID":"6457cb80-cd0c-4342-8fab-4ad86a56af24","Type":"ContainerStarted","Data":"528353ade5e379d0904cc528b48a75f82f02b9ef99985fa3370d4a4025f1900f"} Jul 28 17:10:19.845021 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:19.844989 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" event={"ID":"ed8e1716-5cda-4a93-b714-15d88545f8bd","Type":"ContainerStarted","Data":"7118e9f79910db11602550e57a0742e177145c5b82300c6d9ce35deab62b3d9c"} Jul 28 17:10:19.867134 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:19.867094 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qslrz" event={"ID":"d9ae751d-09fa-4166-b47b-0e069281d796","Type":"ContainerStarted","Data":"e4248fe57201f451d6837b05fd5956424256e530381db60d251f2d9a2a84bf1d"} Jul 28 17:10:19.894685 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:19.889994 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vpsrj" event={"ID":"56e96be9-8647-407e-b2d9-f6718d894258","Type":"ContainerStarted","Data":"2720c77c8b55c397818ee66a96355cca52b9cd888e8dd0d02b0769601bf50ef3"} Jul 28 17:10:19.908661 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:19.908613 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" event={"ID":"536d6825-43fe-4b71-b9d3-9d73177a2a56","Type":"ContainerStarted","Data":"e21bd7a8f6a31a2dfbd9ac4772bcb1e590d32c05615e8334288644be84aa564c"} Jul 28 17:10:20.920742 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:20.920655 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-14.ec2.internal" event={"ID":"1b52734c50b11119e9322d7d16760bf3","Type":"ContainerStarted","Data":"fd14cff68148ce4534ae5796e46365a6cc0bb71aecef0d4e3ff54525757c002d"} Jul 28 17:10:21.343103 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:21.343054 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs\") pod \"network-metrics-daemon-4l6cw\" (UID: \"6cb7d969-3477-425b-9bf1-3192c9d7d94c\") " pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:21.343316 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:21.343128 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret\") pod \"global-pull-secret-syncer-gzjsf\" (UID: \"143142e3-d09f-4dde-bf59-1fffafc3e2fc\") " pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:21.343316 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:21.343280 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:21.343452 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:21.343349 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret podName:143142e3-d09f-4dde-bf59-1fffafc3e2fc nodeName:}" failed. No retries permitted until 2026-07-28 17:10:25.343329173 +0000 UTC m=+10.071830490 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret") pod "global-pull-secret-syncer-gzjsf" (UID: "143142e3-d09f-4dde-bf59-1fffafc3e2fc") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:21.343848 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:21.343798 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:21.343931 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:21.343860 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs podName:6cb7d969-3477-425b-9bf1-3192c9d7d94c nodeName:}" failed. No retries permitted until 2026-07-28 17:10:25.343842458 +0000 UTC m=+10.072343775 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs") pod "network-metrics-daemon-4l6cw" (UID: "6cb7d969-3477-425b-9bf1-3192c9d7d94c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:21.444875 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:21.444141 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzr4x\" (UniqueName: \"kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x\") pod \"network-check-target-kh2q2\" (UID: \"50d07fcb-69de-4546-a128-9549657dd856\") " pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:21.444875 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:21.444376 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:10:21.444875 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:21.444398 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:10:21.444875 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:21.444412 2550 projected.go:196] Error preparing data for projected volume kube-api-access-tzr4x for pod openshift-network-diagnostics/network-check-target-kh2q2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:21.444875 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:21.444494 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x podName:50d07fcb-69de-4546-a128-9549657dd856 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:25.444474069 +0000 UTC m=+10.172975408 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzr4x" (UniqueName: "kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x") pod "network-check-target-kh2q2" (UID: "50d07fcb-69de-4546-a128-9549657dd856") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:21.766631 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:21.766537 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:21.766808 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:21.766673 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:21.766808 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:21.766684 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kh2q2" podUID="50d07fcb-69de-4546-a128-9549657dd856" Jul 28 17:10:21.766808 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:21.766550 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:21.766977 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:21.766807 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4l6cw" podUID="6cb7d969-3477-425b-9bf1-3192c9d7d94c" Jul 28 17:10:21.766977 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:21.766929 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gzjsf" podUID="143142e3-d09f-4dde-bf59-1fffafc3e2fc" Jul 28 17:10:23.766560 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:23.766521 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:23.767028 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:23.766521 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:23.767028 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:23.766636 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:23.767028 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:23.766664 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gzjsf" podUID="143142e3-d09f-4dde-bf59-1fffafc3e2fc" Jul 28 17:10:23.767028 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:23.766735 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kh2q2" podUID="50d07fcb-69de-4546-a128-9549657dd856" Jul 28 17:10:23.767028 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:23.766823 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4l6cw" podUID="6cb7d969-3477-425b-9bf1-3192c9d7d94c" Jul 28 17:10:25.378415 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:25.378249 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs\") pod \"network-metrics-daemon-4l6cw\" (UID: \"6cb7d969-3477-425b-9bf1-3192c9d7d94c\") " pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:25.378415 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:25.378306 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret\") pod \"global-pull-secret-syncer-gzjsf\" (UID: \"143142e3-d09f-4dde-bf59-1fffafc3e2fc\") " pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:25.378922 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:25.378567 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:25.378922 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:25.378632 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs podName:6cb7d969-3477-425b-9bf1-3192c9d7d94c nodeName:}" failed. No retries permitted until 2026-07-28 17:10:33.378611544 +0000 UTC m=+18.107112876 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs") pod "network-metrics-daemon-4l6cw" (UID: "6cb7d969-3477-425b-9bf1-3192c9d7d94c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:25.379132 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:25.379098 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:25.379241 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:25.379166 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret podName:143142e3-d09f-4dde-bf59-1fffafc3e2fc nodeName:}" failed. No retries permitted until 2026-07-28 17:10:33.379148809 +0000 UTC m=+18.107650122 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret") pod "global-pull-secret-syncer-gzjsf" (UID: "143142e3-d09f-4dde-bf59-1fffafc3e2fc") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:25.479103 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:25.478987 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzr4x\" (UniqueName: \"kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x\") pod \"network-check-target-kh2q2\" (UID: \"50d07fcb-69de-4546-a128-9549657dd856\") " pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:25.479255 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:25.479151 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:10:25.479255 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:25.479171 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:10:25.479255 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:25.479184 2550 projected.go:196] Error preparing data for projected volume kube-api-access-tzr4x for pod openshift-network-diagnostics/network-check-target-kh2q2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:25.479255 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:25.479244 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x podName:50d07fcb-69de-4546-a128-9549657dd856 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:33.479227725 +0000 UTC m=+18.207729054 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzr4x" (UniqueName: "kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x") pod "network-check-target-kh2q2" (UID: "50d07fcb-69de-4546-a128-9549657dd856") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:25.768656 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:25.767811 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:25.768656 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:25.767940 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kh2q2" podUID="50d07fcb-69de-4546-a128-9549657dd856" Jul 28 17:10:25.768656 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:25.768316 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:25.768656 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:25.768437 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gzjsf" podUID="143142e3-d09f-4dde-bf59-1fffafc3e2fc" Jul 28 17:10:25.768656 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:25.768488 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:25.768656 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:25.768573 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4l6cw" podUID="6cb7d969-3477-425b-9bf1-3192c9d7d94c" Jul 28 17:10:27.766940 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:27.766901 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:27.767467 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:27.767045 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kh2q2" podUID="50d07fcb-69de-4546-a128-9549657dd856" Jul 28 17:10:27.767467 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:27.767075 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:27.767467 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:27.767199 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gzjsf" podUID="143142e3-d09f-4dde-bf59-1fffafc3e2fc" Jul 28 17:10:27.767467 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:27.767237 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:27.767467 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:27.767329 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4l6cw" podUID="6cb7d969-3477-425b-9bf1-3192c9d7d94c" Jul 28 17:10:29.766755 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:29.766724 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:29.767252 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:29.766719 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:29.767252 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:29.766857 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:29.767252 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:29.766850 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gzjsf" podUID="143142e3-d09f-4dde-bf59-1fffafc3e2fc" Jul 28 17:10:29.767252 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:29.766977 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kh2q2" podUID="50d07fcb-69de-4546-a128-9549657dd856" Jul 28 17:10:29.767252 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:29.767111 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4l6cw" podUID="6cb7d969-3477-425b-9bf1-3192c9d7d94c" Jul 28 17:10:31.766623 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:31.766586 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:31.767126 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:31.766586 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:31.767126 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:31.766746 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:31.767126 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:31.766766 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kh2q2" podUID="50d07fcb-69de-4546-a128-9549657dd856" Jul 28 17:10:31.767126 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:31.766854 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gzjsf" podUID="143142e3-d09f-4dde-bf59-1fffafc3e2fc" Jul 28 17:10:31.767126 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:31.766965 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4l6cw" podUID="6cb7d969-3477-425b-9bf1-3192c9d7d94c" Jul 28 17:10:33.442094 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:33.442048 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs\") pod \"network-metrics-daemon-4l6cw\" (UID: \"6cb7d969-3477-425b-9bf1-3192c9d7d94c\") " pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:33.442627 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:33.442106 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret\") pod \"global-pull-secret-syncer-gzjsf\" (UID: \"143142e3-d09f-4dde-bf59-1fffafc3e2fc\") " pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:33.442627 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:33.442203 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:33.442627 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:33.442263 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs podName:6cb7d969-3477-425b-9bf1-3192c9d7d94c nodeName:}" failed. No retries permitted until 2026-07-28 17:10:49.44224837 +0000 UTC m=+34.170749684 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs") pod "network-metrics-daemon-4l6cw" (UID: "6cb7d969-3477-425b-9bf1-3192c9d7d94c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:33.442627 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:33.442203 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:33.442627 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:33.442307 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret podName:143142e3-d09f-4dde-bf59-1fffafc3e2fc nodeName:}" failed. No retries permitted until 2026-07-28 17:10:49.442297225 +0000 UTC m=+34.170798538 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret") pod "global-pull-secret-syncer-gzjsf" (UID: "143142e3-d09f-4dde-bf59-1fffafc3e2fc") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:33.543094 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:33.543050 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzr4x\" (UniqueName: \"kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x\") pod \"network-check-target-kh2q2\" (UID: \"50d07fcb-69de-4546-a128-9549657dd856\") " pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:33.543292 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:33.543244 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:10:33.543292 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:33.543275 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:10:33.543292 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:33.543288 2550 projected.go:196] Error preparing data for projected volume kube-api-access-tzr4x for pod openshift-network-diagnostics/network-check-target-kh2q2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:33.543485 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:33.543357 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x podName:50d07fcb-69de-4546-a128-9549657dd856 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:49.543337018 +0000 UTC m=+34.271838349 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzr4x" (UniqueName: "kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x") pod "network-check-target-kh2q2" (UID: "50d07fcb-69de-4546-a128-9549657dd856") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:33.766361 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:33.766276 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:33.766533 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:33.766276 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:33.766533 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:33.766440 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kh2q2" podUID="50d07fcb-69de-4546-a128-9549657dd856" Jul 28 17:10:33.766533 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:33.766276 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:33.766533 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:33.766522 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gzjsf" podUID="143142e3-d09f-4dde-bf59-1fffafc3e2fc" Jul 28 17:10:33.766740 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:33.766599 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4l6cw" podUID="6cb7d969-3477-425b-9bf1-3192c9d7d94c" Jul 28 17:10:35.767228 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:35.767188 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:35.767716 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:35.767302 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:35.767716 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:35.767345 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:35.767716 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:35.767350 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4l6cw" podUID="6cb7d969-3477-425b-9bf1-3192c9d7d94c" Jul 28 17:10:35.767716 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:35.767435 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kh2q2" podUID="50d07fcb-69de-4546-a128-9549657dd856" Jul 28 17:10:35.767716 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:35.767506 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gzjsf" podUID="143142e3-d09f-4dde-bf59-1fffafc3e2fc" Jul 28 17:10:37.766925 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:37.766667 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:37.767659 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:37.766756 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:37.767659 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:37.766982 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:37.767659 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:37.766976 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kh2q2" podUID="50d07fcb-69de-4546-a128-9549657dd856" Jul 28 17:10:37.767659 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:37.767152 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gzjsf" podUID="143142e3-d09f-4dde-bf59-1fffafc3e2fc" Jul 28 17:10:37.767659 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:37.767240 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4l6cw" podUID="6cb7d969-3477-425b-9bf1-3192c9d7d94c" Jul 28 17:10:37.959018 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:37.958810 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jblj2" event={"ID":"c05afc7b-dee8-4de5-956e-336b79ac9867","Type":"ContainerStarted","Data":"773bd1766b2a2ec92f3d5394c19a84b5362fa17979a62e10a28272252e7796a7"} Jul 28 17:10:37.960300 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:37.960271 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-x45p5" event={"ID":"c55b772a-3564-4bbe-b57d-073f7d731c17","Type":"ContainerStarted","Data":"553bfffb88c5486be6edc53dc34c20839201a3eb5b2268f4112c4db51a9d065f"} Jul 28 17:10:37.961607 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:37.961583 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4428b" event={"ID":"1dca65a6-b651-46fd-a9ef-fc4eb8ff667e","Type":"ContainerStarted","Data":"31f201cf6aba9ffb590c89f1aeba36cd1d90a8f843dd6104e0963cca073b422c"} Jul 28 17:10:37.962978 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:37.962949 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-g6hlp" event={"ID":"6457cb80-cd0c-4342-8fab-4ad86a56af24","Type":"ContainerStarted","Data":"2d877b1bdc876aa41e11242c9eb148234b7ad174097e0c27576874d52abfa903"} Jul 28 17:10:37.964505 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:37.964484 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" event={"ID":"ed8e1716-5cda-4a93-b714-15d88545f8bd","Type":"ContainerStarted","Data":"36158ca35fff9c0c97d2ed21b2f7296dbbad58100849a5b58e52a1cddb43ec81"} Jul 28 17:10:37.964575 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:37.964539 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" event={"ID":"ed8e1716-5cda-4a93-b714-15d88545f8bd","Type":"ContainerStarted","Data":"86373951455acac065ab200057d308f49c951f29741a73fd577a279db7094d48"} Jul 28 17:10:37.965799 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:37.965781 2550 generic.go:356] "Generic (PLEG): container finished" podID="d9ae751d-09fa-4166-b47b-0e069281d796" containerID="2fee35d93f52581db369d98b5b5917ae21d85f2822edc1aa3e8d7f587ce6d0bc" exitCode=0 Jul 28 17:10:37.965869 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:37.965842 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qslrz" event={"ID":"d9ae751d-09fa-4166-b47b-0e069281d796","Type":"ContainerDied","Data":"2fee35d93f52581db369d98b5b5917ae21d85f2822edc1aa3e8d7f587ce6d0bc"} Jul 28 17:10:37.967369 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:37.967335 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vpsrj" event={"ID":"56e96be9-8647-407e-b2d9-f6718d894258","Type":"ContainerStarted","Data":"0faaa663f22f0b8d6a7c664af8688584a0cfc765f1a7ccb89ae4d15ab62a2c8e"} Jul 28 17:10:37.968830 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:37.968808 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" event={"ID":"536d6825-43fe-4b71-b9d3-9d73177a2a56","Type":"ContainerStarted","Data":"9702734a661b992776070a8d22b55a606f2dd5e4deef7b7a2c10e45c7a2fabdf"} Jul 28 17:10:37.976800 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:37.976755 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jblj2" podStartSLOduration=3.217140273 podStartE2EDuration="21.976740572s" totalImagesPullingTime="18.759600299s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:37.976132903 +0000 UTC m=+22.704634240" watchObservedRunningTime="2026-07-28 17:10:37.976740572 +0000 UTC m=+22.705241908" Jul 28 17:10:37.977095 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:37.977072 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-14.ec2.internal" podStartSLOduration=20.977065471 podStartE2EDuration="20.977065471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:20.944392141 +0000 UTC m=+5.672893499" watchObservedRunningTime="2026-07-28 17:10:37.977065471 +0000 UTC m=+22.705566806" Jul 28 17:10:38.001687 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:38.001638 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vpsrj" podStartSLOduration=3.246493145 podStartE2EDuration="22.001623375s" totalImagesPullingTime="18.75513023s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:38.001268509 +0000 UTC m=+22.729769858" watchObservedRunningTime="2026-07-28 17:10:38.001623375 +0000 UTC m=+22.730124711" Jul 28 17:10:38.045810 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:38.045767 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4428b" podStartSLOduration=3.238046952 podStartE2EDuration="22.045752649s" totalImagesPullingTime="18.807705697s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:38.045226598 +0000 UTC m=+22.773727933" watchObservedRunningTime="2026-07-28 17:10:38.045752649 +0000 UTC m=+22.774253985" Jul 28 17:10:38.064193 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:38.064156 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-g6hlp" podStartSLOduration=3.312440419 podStartE2EDuration="22.064139472s" totalImagesPullingTime="18.751699053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:10:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:38.063996642 +0000 UTC m=+22.792497979" watchObservedRunningTime="2026-07-28 17:10:38.064139472 +0000 UTC m=+22.792640906" Jul 28 17:10:38.976349 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:38.976312 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" event={"ID":"ed8e1716-5cda-4a93-b714-15d88545f8bd","Type":"ContainerStarted","Data":"4f8efb7cd98caa215a344455d7248b6c81f3e494b1a99e2628531b7eef77fab5"} Jul 28 17:10:38.976961 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:38.976414 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" event={"ID":"ed8e1716-5cda-4a93-b714-15d88545f8bd","Type":"ContainerStarted","Data":"d180836f8fdc2f81d2716f3691016d8171ddd2ba87f05ff2cc1e01908a167b1a"} Jul 28 17:10:38.976961 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:38.976464 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" event={"ID":"ed8e1716-5cda-4a93-b714-15d88545f8bd","Type":"ContainerStarted","Data":"f755407fd0aa7cb3ffa564edcbdce49597a25836971b6a8ead55f7a61b32301a"} Jul 28 17:10:38.976961 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:38.976514 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" event={"ID":"ed8e1716-5cda-4a93-b714-15d88545f8bd","Type":"ContainerStarted","Data":"786ceb2ea8f6cd35d68d48f33ec22e4247568ad8e7eba397f8627700c817995f"} Jul 28 17:10:39.324914 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:39.324878 2550 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 28 17:10:39.661657 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:39.661548 2550 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T17:10:39.324906446Z","UUID":"88e209ad-2e7c-4b75-a941-a39e68d39e62","Handler":null,"Name":"","Endpoint":""} Jul 28 17:10:39.663153 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:39.663122 2550 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 28 17:10:39.663288 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:39.663171 2550 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 28 17:10:39.766841 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:39.766812 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:39.767031 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:39.766813 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:39.767031 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:39.766937 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kh2q2" podUID="50d07fcb-69de-4546-a128-9549657dd856" Jul 28 17:10:39.767120 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:39.766812 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:39.767120 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:39.767034 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4l6cw" podUID="6cb7d969-3477-425b-9bf1-3192c9d7d94c" Jul 28 17:10:39.767213 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:39.767143 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gzjsf" podUID="143142e3-d09f-4dde-bf59-1fffafc3e2fc" Jul 28 17:10:39.981345 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:39.981262 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" event={"ID":"536d6825-43fe-4b71-b9d3-9d73177a2a56","Type":"ContainerStarted","Data":"0e6c325f1fe942d3a596c795c1b6d535f8cd8dfe90a6b0e7a6e25dcbe260ea3d"} Jul 28 17:10:39.982926 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:39.982897 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f5tgb" event={"ID":"491ac4f3-9300-4870-bf6e-d48f1c35a53f","Type":"ContainerStarted","Data":"fa2a3816da7522fd23de0dd1c61e9df84853f0694c4cd65e937b71fe8d57300b"} Jul 28 17:10:40.008067 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:40.008013 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-f5tgb" podStartSLOduration=5.249847562 podStartE2EDuration="24.007997943s" totalImagesPullingTime="18.758150381s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:40.007586258 +0000 UTC m=+24.736087595" watchObservedRunningTime="2026-07-28 17:10:40.007997943 +0000 UTC m=+24.736499278" Jul 28 17:10:40.008246 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:40.008212 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-x45p5" podStartSLOduration=5.249019133 podStartE2EDuration="24.008205906s" totalImagesPullingTime="18.759186773s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:38.089576834 +0000 UTC m=+22.818078169" watchObservedRunningTime="2026-07-28 17:10:40.008205906 +0000 UTC m=+24.736707250" Jul 28 17:10:41.766213 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:41.766177 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:41.766703 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:41.766177 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:41.766703 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:41.766315 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gzjsf" podUID="143142e3-d09f-4dde-bf59-1fffafc3e2fc" Jul 28 17:10:41.766703 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:41.766177 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:41.766703 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:41.766401 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kh2q2" podUID="50d07fcb-69de-4546-a128-9549657dd856" Jul 28 17:10:41.766703 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:41.766474 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4l6cw" podUID="6cb7d969-3477-425b-9bf1-3192c9d7d94c" Jul 28 17:10:42.634936 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:42.634651 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-g6hlp" Jul 28 17:10:42.635321 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:42.635291 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-g6hlp" Jul 28 17:10:42.991140 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:42.991108 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" event={"ID":"536d6825-43fe-4b71-b9d3-9d73177a2a56","Type":"ContainerStarted","Data":"9db8966eacb66b3fb3611735df78029a1d93f8b828f4cc2d39f502e298206bd8"} Jul 28 17:10:42.994244 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:42.994221 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" event={"ID":"ed8e1716-5cda-4a93-b714-15d88545f8bd","Type":"ContainerStarted","Data":"d94dc906ea8650d7b7021e70e765ed1e701797ce75c48e2c559d25f31cb5d2ec"} Jul 28 17:10:42.995929 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:42.995905 2550 generic.go:356] "Generic (PLEG): container finished" podID="d9ae751d-09fa-4166-b47b-0e069281d796" containerID="c3c5463c33da67c838149ca160efcb44e66fbedc0519f0759dbc8b499418a529" exitCode=0 Jul 28 17:10:42.996017 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:42.995986 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qslrz" event={"ID":"d9ae751d-09fa-4166-b47b-0e069281d796","Type":"ContainerDied","Data":"c3c5463c33da67c838149ca160efcb44e66fbedc0519f0759dbc8b499418a529"} Jul 28 17:10:42.996220 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:42.996191 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-g6hlp" Jul 28 17:10:42.996627 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:42.996571 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-g6hlp" Jul 28 17:10:43.016643 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:43.016586 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-j2hkp" podStartSLOduration=3.6822064169999997 podStartE2EDuration="27.0165715s" totalImagesPullingTime="23.334365083s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:43.015873064 +0000 UTC m=+27.744374401" watchObservedRunningTime="2026-07-28 17:10:43.0165715 +0000 UTC m=+27.745072830" Jul 28 17:10:43.766691 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:43.766649 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:43.766865 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:43.766649 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:43.766865 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:43.766788 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4l6cw" podUID="6cb7d969-3477-425b-9bf1-3192c9d7d94c" Jul 28 17:10:43.766865 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:43.766835 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:43.766991 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:43.766921 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kh2q2" podUID="50d07fcb-69de-4546-a128-9549657dd856" Jul 28 17:10:43.767036 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:43.767000 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gzjsf" podUID="143142e3-d09f-4dde-bf59-1fffafc3e2fc" Jul 28 17:10:45.004438 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:45.004044 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" event={"ID":"ed8e1716-5cda-4a93-b714-15d88545f8bd","Type":"ContainerStarted","Data":"bba92f75433f3b01219d13822e109fcd55e5ec3b6a205a72dbc0438ebf4b655c"} Jul 28 17:10:45.004438 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:45.004322 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:45.004438 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:45.004373 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:45.004438 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:45.004430 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:45.019792 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:45.019764 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:45.019917 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:45.019886 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:10:45.057823 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:45.057765 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" podStartSLOduration=10.239344355 podStartE2EDuration="29.057748519s" totalImagesPullingTime="18.818404164s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:45.055957777 +0000 UTC m=+29.784459114" watchObservedRunningTime="2026-07-28 17:10:45.057748519 +0000 UTC m=+29.786249856" Jul 28 17:10:45.767293 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:45.767257 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:45.767483 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:45.767363 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:45.767483 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:45.767370 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4l6cw" podUID="6cb7d969-3477-425b-9bf1-3192c9d7d94c" Jul 28 17:10:45.767483 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:45.767458 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gzjsf" podUID="143142e3-d09f-4dde-bf59-1fffafc3e2fc" Jul 28 17:10:45.767617 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:45.767486 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:45.767617 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:45.767534 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kh2q2" podUID="50d07fcb-69de-4546-a128-9549657dd856" Jul 28 17:10:46.009075 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:46.009034 2550 generic.go:356] "Generic (PLEG): container finished" podID="d9ae751d-09fa-4166-b47b-0e069281d796" containerID="30dc49f1947fa2faa009dbf05baf1a008f3f27e94a1743525246f1b0e331fde7" exitCode=0 Jul 28 17:10:46.009603 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:46.009154 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qslrz" event={"ID":"d9ae751d-09fa-4166-b47b-0e069281d796","Type":"ContainerDied","Data":"30dc49f1947fa2faa009dbf05baf1a008f3f27e94a1743525246f1b0e331fde7"} Jul 28 17:10:46.879998 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:46.879958 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kh2q2"] Jul 28 17:10:46.880323 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:46.880137 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:46.880323 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:46.880241 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kh2q2" podUID="50d07fcb-69de-4546-a128-9549657dd856" Jul 28 17:10:46.882314 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:46.882289 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gzjsf"] Jul 28 17:10:46.882444 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:46.882413 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:46.882557 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:46.882528 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gzjsf" podUID="143142e3-d09f-4dde-bf59-1fffafc3e2fc" Jul 28 17:10:46.884939 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:46.884914 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4l6cw"] Jul 28 17:10:46.885070 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:46.885060 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:46.885213 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:46.885173 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4l6cw" podUID="6cb7d969-3477-425b-9bf1-3192c9d7d94c" Jul 28 17:10:48.017650 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:48.017460 2550 generic.go:356] "Generic (PLEG): container finished" podID="d9ae751d-09fa-4166-b47b-0e069281d796" containerID="e19a61ac7aba858cfed43889a43238269b2c9668ecced09b36706ccdffd33488" exitCode=0 Jul 28 17:10:48.018114 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:48.017545 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qslrz" event={"ID":"d9ae751d-09fa-4166-b47b-0e069281d796","Type":"ContainerDied","Data":"e19a61ac7aba858cfed43889a43238269b2c9668ecced09b36706ccdffd33488"} Jul 28 17:10:48.766294 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:48.766221 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:48.766468 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:48.766221 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:48.766468 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:48.766342 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gzjsf" podUID="143142e3-d09f-4dde-bf59-1fffafc3e2fc" Jul 28 17:10:48.766468 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:48.766223 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:48.766468 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:48.766424 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kh2q2" podUID="50d07fcb-69de-4546-a128-9549657dd856" Jul 28 17:10:48.766681 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:48.766496 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4l6cw" podUID="6cb7d969-3477-425b-9bf1-3192c9d7d94c" Jul 28 17:10:49.459250 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:49.459213 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs\") pod \"network-metrics-daemon-4l6cw\" (UID: \"6cb7d969-3477-425b-9bf1-3192c9d7d94c\") " pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:49.459770 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:49.459258 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret\") pod \"global-pull-secret-syncer-gzjsf\" (UID: \"143142e3-d09f-4dde-bf59-1fffafc3e2fc\") " pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:49.459770 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:49.459404 2550 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:49.459770 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:49.459476 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret podName:143142e3-d09f-4dde-bf59-1fffafc3e2fc nodeName:}" failed. No retries permitted until 2026-07-28 17:11:21.459455004 +0000 UTC m=+66.187956321 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret") pod "global-pull-secret-syncer-gzjsf" (UID: "143142e3-d09f-4dde-bf59-1fffafc3e2fc") : object "kube-system"/"original-pull-secret" not registered Jul 28 17:10:49.459770 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:49.459404 2550 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:49.459770 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:49.459543 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs podName:6cb7d969-3477-425b-9bf1-3192c9d7d94c nodeName:}" failed. No retries permitted until 2026-07-28 17:11:21.459529186 +0000 UTC m=+66.188030511 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs") pod "network-metrics-daemon-4l6cw" (UID: "6cb7d969-3477-425b-9bf1-3192c9d7d94c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 28 17:10:49.560603 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:49.560514 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzr4x\" (UniqueName: \"kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x\") pod \"network-check-target-kh2q2\" (UID: \"50d07fcb-69de-4546-a128-9549657dd856\") " pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:49.560761 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:49.560692 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 28 17:10:49.560761 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:49.560712 2550 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 28 17:10:49.560761 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:49.560722 2550 projected.go:196] Error preparing data for projected volume kube-api-access-tzr4x for pod openshift-network-diagnostics/network-check-target-kh2q2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:49.560935 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:49.560785 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x podName:50d07fcb-69de-4546-a128-9549657dd856 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:21.560772397 +0000 UTC m=+66.289273714 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzr4x" (UniqueName: "kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x") pod "network-check-target-kh2q2" (UID: "50d07fcb-69de-4546-a128-9549657dd856") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 28 17:10:50.766849 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:50.766810 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:50.767459 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:50.766810 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:50.767459 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:50.766951 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gzjsf" podUID="143142e3-d09f-4dde-bf59-1fffafc3e2fc" Jul 28 17:10:50.767459 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:50.766810 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:50.767459 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:50.767010 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kh2q2" podUID="50d07fcb-69de-4546-a128-9549657dd856" Jul 28 17:10:50.767459 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:50.767104 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4l6cw" podUID="6cb7d969-3477-425b-9bf1-3192c9d7d94c" Jul 28 17:10:51.101908 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.101880 2550 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-14.ec2.internal" event="NodeReady" Jul 28 17:10:51.102094 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.102013 2550 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 28 17:10:51.179688 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.179652 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5547bf85cb-jklcq"] Jul 28 17:10:51.184647 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.184625 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.192747 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:51.192717 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"registry-dockercfg-r5xqg\" is forbidden: User \"system:node:ip-10-0-134-14.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-134-14.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-r5xqg\"" type="*v1.Secret" Jul 28 17:10:51.192747 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:51.192719 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"image-registry-tls\" is forbidden: User \"system:node:ip-10-0-134-14.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-134-14.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" type="*v1.Secret" Jul 28 17:10:51.194530 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:51.194496 2550 status_manager.go:1018] "Failed to get status for pod" err="pods \"image-registry-5547bf85cb-jklcq\" is forbidden: User \"system:node:ip-10-0-134-14.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-134-14.ec2.internal' and this object" podUID="39309997-adcd-4fb1-94d4-ac4668989199" pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.195183 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:51.194929 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"image-registry-private-configuration\" is forbidden: User \"system:node:ip-10-0-134-14.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-134-14.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" type="*v1.Secret" Jul 28 17:10:51.195294 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:51.195260 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"trusted-ca\" is forbidden: User \"system:node:ip-10-0-134-14.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-134-14.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" type="*v1.ConfigMap" Jul 28 17:10:51.195294 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:51.195268 2550 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"installation-pull-secrets\" is forbidden: User \"system:node:ip-10-0-134-14.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-image-registry\": no relationship found between node 'ip-10-0-134-14.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" type="*v1.Secret" Jul 28 17:10:51.219716 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.219615 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5547bf85cb-jklcq"] Jul 28 17:10:51.221042 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.220342 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fxdhd"] Jul 28 17:10:51.225058 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.225033 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-xv5f2"] Jul 28 17:10:51.225184 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.225165 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fxdhd" Jul 28 17:10:51.228788 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.228764 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-hzlqt"] Jul 28 17:10:51.228908 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.228876 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xv5f2" Jul 28 17:10:51.232638 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.232245 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp"] Jul 28 17:10:51.232638 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.232341 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-hzlqt" Jul 28 17:10:51.235643 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.235623 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-9whj6"] Jul 28 17:10:51.235768 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.235752 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp" Jul 28 17:10:51.237036 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.237014 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 28 17:10:51.237036 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.237040 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 28 17:10:51.237193 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.237015 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 28 17:10:51.237193 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.237100 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-5c84r\"" Jul 28 17:10:51.238218 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.238202 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 28 17:10:51.238621 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.238601 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:10:51.238984 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.238965 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9whj6" Jul 28 17:10:51.239550 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.239533 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 28 17:10:51.241714 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.241542 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cpt5q\"" Jul 28 17:10:51.248287 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.247309 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 28 17:10:51.249712 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.249686 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-hzlqt"] Jul 28 17:10:51.252280 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.250540 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-t84jj\"" Jul 28 17:10:51.252280 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.250645 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 28 17:10:51.252280 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.251225 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 28 17:10:51.252280 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.251268 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 28 17:10:51.252280 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.251915 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 28 17:10:51.252280 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.252143 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 28 17:10:51.252280 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.252248 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-6msjb\"" Jul 28 17:10:51.253085 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.252763 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 28 17:10:51.253085 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.252987 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-2m7qb\"" Jul 28 17:10:51.253085 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.253012 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-xv5f2"] Jul 28 17:10:51.255239 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.253514 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 28 17:10:51.255239 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.254319 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fxdhd"] Jul 28 17:10:51.256073 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.256054 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-9whj6"] Jul 28 17:10:51.269668 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.269635 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 28 17:10:51.270061 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.270041 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp"] Jul 28 17:10:51.275158 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.275097 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-bound-sa-token\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.275262 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.275168 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bfpd\" (UniqueName: \"kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-kube-api-access-9bfpd\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.275262 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.275189 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39309997-adcd-4fb1-94d4-ac4668989199-ca-trust-extracted\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.275262 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.275210 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39309997-adcd-4fb1-94d4-ac4668989199-image-registry-private-configuration\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.275262 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.275246 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39309997-adcd-4fb1-94d4-ac4668989199-registry-certificates\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.275445 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.275278 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-registry-tls\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.275445 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.275305 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39309997-adcd-4fb1-94d4-ac4668989199-trusted-ca\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.275445 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.275335 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39309997-adcd-4fb1-94d4-ac4668989199-installation-pull-secrets\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.317651 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.317615 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jx7cl"] Jul 28 17:10:51.321680 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.321653 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jx7cl" Jul 28 17:10:51.324987 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.324953 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 28 17:10:51.325517 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.325494 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 28 17:10:51.325634 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.325543 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hqvdh\"" Jul 28 17:10:51.342751 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.342721 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jx7cl"] Jul 28 17:10:51.376078 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.376041 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-registry-tls\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.376264 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.376090 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5a580271-8c91-43b9-8479-21fedddcf96f-serving-cert\") pod \"service-ca-operator-5d6c75f84f-hzlqt\" (UID: \"5a580271-8c91-43b9-8479-21fedddcf96f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-hzlqt" Jul 28 17:10:51.376264 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.376117 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39309997-adcd-4fb1-94d4-ac4668989199-trusted-ca\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.376264 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.376157 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39309997-adcd-4fb1-94d4-ac4668989199-installation-pull-secrets\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.376264 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.376188 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjfp8\" (UniqueName: \"kubernetes.io/projected/6f52b7f0-cbf9-46aa-bac6-b7fd7486ea50-kube-api-access-mjfp8\") pod \"ingress-canary-fxdhd\" (UID: \"6f52b7f0-cbf9-46aa-bac6-b7fd7486ea50\") " pod="openshift-ingress-canary/ingress-canary-fxdhd" Jul 28 17:10:51.376264 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.376206 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-bound-sa-token\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.376264 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.376228 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-9whj6\" (UID: \"8cc0aa75-c527-429b-9acd-37b1b1f9c6c7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9whj6" Jul 28 17:10:51.376563 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.376299 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9whj6\" (UID: \"8cc0aa75-c527-429b-9acd-37b1b1f9c6c7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9whj6" Jul 28 17:10:51.376563 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.376369 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/73e0baae-ba8a-4ef3-aff6-a31782400af6-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp\" (UID: \"73e0baae-ba8a-4ef3-aff6-a31782400af6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp" Jul 28 17:10:51.376563 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.376412 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/73e0baae-ba8a-4ef3-aff6-a31782400af6-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp\" (UID: \"73e0baae-ba8a-4ef3-aff6-a31782400af6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp" Jul 28 17:10:51.376563 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.376438 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f52b7f0-cbf9-46aa-bac6-b7fd7486ea50-cert\") pod \"ingress-canary-fxdhd\" (UID: \"6f52b7f0-cbf9-46aa-bac6-b7fd7486ea50\") " pod="openshift-ingress-canary/ingress-canary-fxdhd" Jul 28 17:10:51.376563 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.376540 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bfpd\" (UniqueName: \"kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-kube-api-access-9bfpd\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.376769 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.376579 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39309997-adcd-4fb1-94d4-ac4668989199-ca-trust-extracted\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.376769 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.376603 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39309997-adcd-4fb1-94d4-ac4668989199-image-registry-private-configuration\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.376769 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.376623 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5a580271-8c91-43b9-8479-21fedddcf96f-config\") pod \"service-ca-operator-5d6c75f84f-hzlqt\" (UID: \"5a580271-8c91-43b9-8479-21fedddcf96f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-hzlqt" Jul 28 17:10:51.376769 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.376644 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn6rz\" (UniqueName: \"kubernetes.io/projected/5a580271-8c91-43b9-8479-21fedddcf96f-kube-api-access-cn6rz\") pod \"service-ca-operator-5d6c75f84f-hzlqt\" (UID: \"5a580271-8c91-43b9-8479-21fedddcf96f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-hzlqt" Jul 28 17:10:51.376769 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.376660 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjsdb\" (UniqueName: \"kubernetes.io/projected/73e0baae-ba8a-4ef3-aff6-a31782400af6-kube-api-access-kjsdb\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp\" (UID: \"73e0baae-ba8a-4ef3-aff6-a31782400af6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp" Jul 28 17:10:51.376769 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.376694 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42z2s\" (UniqueName: \"kubernetes.io/projected/f16ab6fd-caec-4bb9-8aa8-d146b004e239-kube-api-access-42z2s\") pod \"network-check-source-7d88cb55d8-xv5f2\" (UID: \"f16ab6fd-caec-4bb9-8aa8-d146b004e239\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xv5f2" Jul 28 17:10:51.376769 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.376736 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39309997-adcd-4fb1-94d4-ac4668989199-registry-certificates\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.377137 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.377117 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39309997-adcd-4fb1-94d4-ac4668989199-ca-trust-extracted\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.377331 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.377314 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39309997-adcd-4fb1-94d4-ac4668989199-registry-certificates\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.398461 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.398424 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bfpd\" (UniqueName: \"kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-kube-api-access-9bfpd\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.399975 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.399951 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-bound-sa-token\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:51.477866 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.477822 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l8ng\" (UniqueName: \"kubernetes.io/projected/4b6d4302-fb6f-44e7-8b39-07a78c35b277-kube-api-access-6l8ng\") pod \"dns-default-jx7cl\" (UID: \"4b6d4302-fb6f-44e7-8b39-07a78c35b277\") " pod="openshift-dns/dns-default-jx7cl" Jul 28 17:10:51.478043 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.477900 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjfp8\" (UniqueName: \"kubernetes.io/projected/6f52b7f0-cbf9-46aa-bac6-b7fd7486ea50-kube-api-access-mjfp8\") pod \"ingress-canary-fxdhd\" (UID: \"6f52b7f0-cbf9-46aa-bac6-b7fd7486ea50\") " pod="openshift-ingress-canary/ingress-canary-fxdhd" Jul 28 17:10:51.478043 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.477933 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-9whj6\" (UID: \"8cc0aa75-c527-429b-9acd-37b1b1f9c6c7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9whj6" Jul 28 17:10:51.478043 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.477961 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9whj6\" (UID: \"8cc0aa75-c527-429b-9acd-37b1b1f9c6c7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9whj6" Jul 28 17:10:51.478043 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.477987 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/73e0baae-ba8a-4ef3-aff6-a31782400af6-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp\" (UID: \"73e0baae-ba8a-4ef3-aff6-a31782400af6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp" Jul 28 17:10:51.478043 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.478015 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/73e0baae-ba8a-4ef3-aff6-a31782400af6-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp\" (UID: \"73e0baae-ba8a-4ef3-aff6-a31782400af6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp" Jul 28 17:10:51.478043 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.478038 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f52b7f0-cbf9-46aa-bac6-b7fd7486ea50-cert\") pod \"ingress-canary-fxdhd\" (UID: \"6f52b7f0-cbf9-46aa-bac6-b7fd7486ea50\") " pod="openshift-ingress-canary/ingress-canary-fxdhd" Jul 28 17:10:51.478270 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.478062 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b6d4302-fb6f-44e7-8b39-07a78c35b277-config-volume\") pod \"dns-default-jx7cl\" (UID: \"4b6d4302-fb6f-44e7-8b39-07a78c35b277\") " pod="openshift-dns/dns-default-jx7cl" Jul 28 17:10:51.478270 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:51.478090 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 17:10:51.478270 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.478129 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5a580271-8c91-43b9-8479-21fedddcf96f-config\") pod \"service-ca-operator-5d6c75f84f-hzlqt\" (UID: \"5a580271-8c91-43b9-8479-21fedddcf96f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-hzlqt" Jul 28 17:10:51.478270 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:51.478158 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-networking-console-plugin-cert podName:8cc0aa75-c527-429b-9acd-37b1b1f9c6c7 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:51.978136587 +0000 UTC m=+36.706637918 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9whj6" (UID: "8cc0aa75-c527-429b-9acd-37b1b1f9c6c7") : secret "networking-console-plugin-cert" not found Jul 28 17:10:51.478270 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.478193 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn6rz\" (UniqueName: \"kubernetes.io/projected/5a580271-8c91-43b9-8479-21fedddcf96f-kube-api-access-cn6rz\") pod \"service-ca-operator-5d6c75f84f-hzlqt\" (UID: \"5a580271-8c91-43b9-8479-21fedddcf96f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-hzlqt" Jul 28 17:10:51.478270 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.478224 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjsdb\" (UniqueName: \"kubernetes.io/projected/73e0baae-ba8a-4ef3-aff6-a31782400af6-kube-api-access-kjsdb\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp\" (UID: \"73e0baae-ba8a-4ef3-aff6-a31782400af6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp" Jul 28 17:10:51.478509 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.478269 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42z2s\" (UniqueName: \"kubernetes.io/projected/f16ab6fd-caec-4bb9-8aa8-d146b004e239-kube-api-access-42z2s\") pod \"network-check-source-7d88cb55d8-xv5f2\" (UID: \"f16ab6fd-caec-4bb9-8aa8-d146b004e239\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xv5f2" Jul 28 17:10:51.478509 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.478336 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b6d4302-fb6f-44e7-8b39-07a78c35b277-tmp-dir\") pod \"dns-default-jx7cl\" (UID: \"4b6d4302-fb6f-44e7-8b39-07a78c35b277\") " pod="openshift-dns/dns-default-jx7cl" Jul 28 17:10:51.478509 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.478368 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5a580271-8c91-43b9-8479-21fedddcf96f-serving-cert\") pod \"service-ca-operator-5d6c75f84f-hzlqt\" (UID: \"5a580271-8c91-43b9-8479-21fedddcf96f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-hzlqt" Jul 28 17:10:51.478509 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.478419 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b6d4302-fb6f-44e7-8b39-07a78c35b277-metrics-tls\") pod \"dns-default-jx7cl\" (UID: \"4b6d4302-fb6f-44e7-8b39-07a78c35b277\") " pod="openshift-dns/dns-default-jx7cl" Jul 28 17:10:51.478636 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.478625 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5a580271-8c91-43b9-8479-21fedddcf96f-config\") pod \"service-ca-operator-5d6c75f84f-hzlqt\" (UID: \"5a580271-8c91-43b9-8479-21fedddcf96f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-hzlqt" Jul 28 17:10:51.478752 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.478724 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-9whj6\" (UID: \"8cc0aa75-c527-429b-9acd-37b1b1f9c6c7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9whj6" Jul 28 17:10:51.479249 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.479225 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/73e0baae-ba8a-4ef3-aff6-a31782400af6-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp\" (UID: \"73e0baae-ba8a-4ef3-aff6-a31782400af6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp" Jul 28 17:10:51.480858 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.480830 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/73e0baae-ba8a-4ef3-aff6-a31782400af6-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp\" (UID: \"73e0baae-ba8a-4ef3-aff6-a31782400af6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp" Jul 28 17:10:51.481028 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.481002 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5a580271-8c91-43b9-8479-21fedddcf96f-serving-cert\") pod \"service-ca-operator-5d6c75f84f-hzlqt\" (UID: \"5a580271-8c91-43b9-8479-21fedddcf96f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-hzlqt" Jul 28 17:10:51.481439 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.481419 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f52b7f0-cbf9-46aa-bac6-b7fd7486ea50-cert\") pod \"ingress-canary-fxdhd\" (UID: \"6f52b7f0-cbf9-46aa-bac6-b7fd7486ea50\") " pod="openshift-ingress-canary/ingress-canary-fxdhd" Jul 28 17:10:51.495811 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.495786 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn6rz\" (UniqueName: \"kubernetes.io/projected/5a580271-8c91-43b9-8479-21fedddcf96f-kube-api-access-cn6rz\") pod \"service-ca-operator-5d6c75f84f-hzlqt\" (UID: \"5a580271-8c91-43b9-8479-21fedddcf96f\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-hzlqt" Jul 28 17:10:51.499961 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.499904 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjsdb\" (UniqueName: \"kubernetes.io/projected/73e0baae-ba8a-4ef3-aff6-a31782400af6-kube-api-access-kjsdb\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp\" (UID: \"73e0baae-ba8a-4ef3-aff6-a31782400af6\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp" Jul 28 17:10:51.499961 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.499953 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-42z2s\" (UniqueName: \"kubernetes.io/projected/f16ab6fd-caec-4bb9-8aa8-d146b004e239-kube-api-access-42z2s\") pod \"network-check-source-7d88cb55d8-xv5f2\" (UID: \"f16ab6fd-caec-4bb9-8aa8-d146b004e239\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xv5f2" Jul 28 17:10:51.501930 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.501910 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjfp8\" (UniqueName: \"kubernetes.io/projected/6f52b7f0-cbf9-46aa-bac6-b7fd7486ea50-kube-api-access-mjfp8\") pod \"ingress-canary-fxdhd\" (UID: \"6f52b7f0-cbf9-46aa-bac6-b7fd7486ea50\") " pod="openshift-ingress-canary/ingress-canary-fxdhd" Jul 28 17:10:51.536491 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.536460 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fxdhd" Jul 28 17:10:51.543634 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.543608 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xv5f2" Jul 28 17:10:51.554931 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.554906 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-hzlqt" Jul 28 17:10:51.564167 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.564142 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp" Jul 28 17:10:51.578846 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.578819 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b6d4302-fb6f-44e7-8b39-07a78c35b277-config-volume\") pod \"dns-default-jx7cl\" (UID: \"4b6d4302-fb6f-44e7-8b39-07a78c35b277\") " pod="openshift-dns/dns-default-jx7cl" Jul 28 17:10:51.578979 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.578900 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b6d4302-fb6f-44e7-8b39-07a78c35b277-tmp-dir\") pod \"dns-default-jx7cl\" (UID: \"4b6d4302-fb6f-44e7-8b39-07a78c35b277\") " pod="openshift-dns/dns-default-jx7cl" Jul 28 17:10:51.578979 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.578925 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b6d4302-fb6f-44e7-8b39-07a78c35b277-metrics-tls\") pod \"dns-default-jx7cl\" (UID: \"4b6d4302-fb6f-44e7-8b39-07a78c35b277\") " pod="openshift-dns/dns-default-jx7cl" Jul 28 17:10:51.578979 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.578941 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l8ng\" (UniqueName: \"kubernetes.io/projected/4b6d4302-fb6f-44e7-8b39-07a78c35b277-kube-api-access-6l8ng\") pod \"dns-default-jx7cl\" (UID: \"4b6d4302-fb6f-44e7-8b39-07a78c35b277\") " pod="openshift-dns/dns-default-jx7cl" Jul 28 17:10:51.579123 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:51.579082 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:10:51.579182 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:51.579160 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b6d4302-fb6f-44e7-8b39-07a78c35b277-metrics-tls podName:4b6d4302-fb6f-44e7-8b39-07a78c35b277 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:52.079138652 +0000 UTC m=+36.807639975 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b6d4302-fb6f-44e7-8b39-07a78c35b277-metrics-tls") pod "dns-default-jx7cl" (UID: "4b6d4302-fb6f-44e7-8b39-07a78c35b277") : secret "dns-default-metrics-tls" not found Jul 28 17:10:51.579328 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.579304 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4b6d4302-fb6f-44e7-8b39-07a78c35b277-tmp-dir\") pod \"dns-default-jx7cl\" (UID: \"4b6d4302-fb6f-44e7-8b39-07a78c35b277\") " pod="openshift-dns/dns-default-jx7cl" Jul 28 17:10:51.579533 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.579508 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4b6d4302-fb6f-44e7-8b39-07a78c35b277-config-volume\") pod \"dns-default-jx7cl\" (UID: \"4b6d4302-fb6f-44e7-8b39-07a78c35b277\") " pod="openshift-dns/dns-default-jx7cl" Jul 28 17:10:51.593801 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.593774 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l8ng\" (UniqueName: \"kubernetes.io/projected/4b6d4302-fb6f-44e7-8b39-07a78c35b277-kube-api-access-6l8ng\") pod \"dns-default-jx7cl\" (UID: \"4b6d4302-fb6f-44e7-8b39-07a78c35b277\") " pod="openshift-dns/dns-default-jx7cl" Jul 28 17:10:51.984135 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:51.983674 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9whj6\" (UID: \"8cc0aa75-c527-429b-9acd-37b1b1f9c6c7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9whj6" Jul 28 17:10:51.984783 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:51.984138 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 17:10:51.984783 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:51.984202 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-networking-console-plugin-cert podName:8cc0aa75-c527-429b-9acd-37b1b1f9c6c7 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:52.984183667 +0000 UTC m=+37.712684999 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9whj6" (UID: "8cc0aa75-c527-429b-9acd-37b1b1f9c6c7") : secret "networking-console-plugin-cert" not found Jul 28 17:10:52.064514 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.064491 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-hzlqt"] Jul 28 17:10:52.067128 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:52.067098 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a580271_8c91_43b9_8479_21fedddcf96f.slice/crio-08a7a5197c4f1dca28e40a4204d4757b71a5ffa2cfa5bb602f938c9999442ebb WatchSource:0}: Error finding container 08a7a5197c4f1dca28e40a4204d4757b71a5ffa2cfa5bb602f938c9999442ebb: Status 404 returned error can't find the container with id 08a7a5197c4f1dca28e40a4204d4757b71a5ffa2cfa5bb602f938c9999442ebb Jul 28 17:10:52.069897 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.069795 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-xv5f2"] Jul 28 17:10:52.075275 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:52.075251 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf16ab6fd_caec_4bb9_8aa8_d146b004e239.slice/crio-119419e9f996b8fcb5642a67951758c92e1c905b2b280272d4c6ddec7a327b35 WatchSource:0}: Error finding container 119419e9f996b8fcb5642a67951758c92e1c905b2b280272d4c6ddec7a327b35: Status 404 returned error can't find the container with id 119419e9f996b8fcb5642a67951758c92e1c905b2b280272d4c6ddec7a327b35 Jul 28 17:10:52.078892 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.078865 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fxdhd"] Jul 28 17:10:52.084649 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.084622 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b6d4302-fb6f-44e7-8b39-07a78c35b277-metrics-tls\") pod \"dns-default-jx7cl\" (UID: \"4b6d4302-fb6f-44e7-8b39-07a78c35b277\") " pod="openshift-dns/dns-default-jx7cl" Jul 28 17:10:52.084781 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:52.084767 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:10:52.084823 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:52.084818 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b6d4302-fb6f-44e7-8b39-07a78c35b277-metrics-tls podName:4b6d4302-fb6f-44e7-8b39-07a78c35b277 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:53.084804389 +0000 UTC m=+37.813305706 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b6d4302-fb6f-44e7-8b39-07a78c35b277-metrics-tls") pod "dns-default-jx7cl" (UID: "4b6d4302-fb6f-44e7-8b39-07a78c35b277") : secret "dns-default-metrics-tls" not found Jul 28 17:10:52.095319 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:52.095293 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f52b7f0_cbf9_46aa_bac6_b7fd7486ea50.slice/crio-630a589fc43bfa5cf9f4f5ee78fe47dcbce4237b78e1446716c101747555ad2a WatchSource:0}: Error finding container 630a589fc43bfa5cf9f4f5ee78fe47dcbce4237b78e1446716c101747555ad2a: Status 404 returned error can't find the container with id 630a589fc43bfa5cf9f4f5ee78fe47dcbce4237b78e1446716c101747555ad2a Jul 28 17:10:52.098976 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.098949 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp"] Jul 28 17:10:52.109649 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:10:52.109620 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73e0baae_ba8a_4ef3_aff6_a31782400af6.slice/crio-11afdcaaae43dbb41097e9bdabf63a5f2087cf6efd7a92530979518c1126472f WatchSource:0}: Error finding container 11afdcaaae43dbb41097e9bdabf63a5f2087cf6efd7a92530979518c1126472f: Status 404 returned error can't find the container with id 11afdcaaae43dbb41097e9bdabf63a5f2087cf6efd7a92530979518c1126472f Jul 28 17:10:52.329050 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.329017 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 28 17:10:52.339965 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.339936 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39309997-adcd-4fb1-94d4-ac4668989199-installation-pull-secrets\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:52.376339 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:52.376295 2550 configmap.go:193] Couldn't get configMap openshift-image-registry/trusted-ca: failed to sync configmap cache: timed out waiting for the condition Jul 28 17:10:52.376533 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:52.376303 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: failed to sync secret cache: timed out waiting for the condition Jul 28 17:10:52.376533 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:52.376419 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5547bf85cb-jklcq: failed to sync secret cache: timed out waiting for the condition Jul 28 17:10:52.376533 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:52.376410 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39309997-adcd-4fb1-94d4-ac4668989199-trusted-ca podName:39309997-adcd-4fb1-94d4-ac4668989199 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:52.876375989 +0000 UTC m=+37.604877316 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "trusted-ca" (UniqueName: "kubernetes.io/configmap/39309997-adcd-4fb1-94d4-ac4668989199-trusted-ca") pod "image-registry-5547bf85cb-jklcq" (UID: "39309997-adcd-4fb1-94d4-ac4668989199") : failed to sync configmap cache: timed out waiting for the condition Jul 28 17:10:52.376533 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:52.376471 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-registry-tls podName:39309997-adcd-4fb1-94d4-ac4668989199 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:52.876456042 +0000 UTC m=+37.604957355 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-registry-tls") pod "image-registry-5547bf85cb-jklcq" (UID: "39309997-adcd-4fb1-94d4-ac4668989199") : failed to sync secret cache: timed out waiting for the condition Jul 28 17:10:52.377414 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:52.377379 2550 secret.go:189] Couldn't get secret openshift-image-registry/image-registry-private-configuration: failed to sync secret cache: timed out waiting for the condition Jul 28 17:10:52.377454 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:52.377445 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39309997-adcd-4fb1-94d4-ac4668989199-image-registry-private-configuration podName:39309997-adcd-4fb1-94d4-ac4668989199 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:52.877434857 +0000 UTC m=+37.605936176 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "image-registry-private-configuration" (UniqueName: "kubernetes.io/secret/39309997-adcd-4fb1-94d4-ac4668989199-image-registry-private-configuration") pod "image-registry-5547bf85cb-jklcq" (UID: "39309997-adcd-4fb1-94d4-ac4668989199") : failed to sync secret cache: timed out waiting for the condition Jul 28 17:10:52.492474 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.492149 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 28 17:10:52.514669 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.514630 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 28 17:10:52.599870 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.599829 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-r5xqg\"" Jul 28 17:10:52.602309 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.602293 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 28 17:10:52.766449 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.766355 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:10:52.766603 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.766498 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:10:52.766603 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.766533 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:10:52.770270 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.770245 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 17:10:52.770474 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.770306 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wz9mb\"" Jul 28 17:10:52.776072 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.776051 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 17:10:52.778694 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.778670 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xf5sm\"" Jul 28 17:10:52.891236 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.891194 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39309997-adcd-4fb1-94d4-ac4668989199-image-registry-private-configuration\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:52.891446 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.891284 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-registry-tls\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:52.891446 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.891319 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39309997-adcd-4fb1-94d4-ac4668989199-trusted-ca\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:52.891791 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:52.891681 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:10:52.891791 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:52.891706 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5547bf85cb-jklcq: secret "image-registry-tls" not found Jul 28 17:10:52.891953 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:52.891797 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-registry-tls podName:39309997-adcd-4fb1-94d4-ac4668989199 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:53.891766111 +0000 UTC m=+38.620267447 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-registry-tls") pod "image-registry-5547bf85cb-jklcq" (UID: "39309997-adcd-4fb1-94d4-ac4668989199") : secret "image-registry-tls" not found Jul 28 17:10:52.892903 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.892877 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39309997-adcd-4fb1-94d4-ac4668989199-trusted-ca\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:52.894669 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.894646 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39309997-adcd-4fb1-94d4-ac4668989199-image-registry-private-configuration\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:52.992457 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:52.992412 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9whj6\" (UID: \"8cc0aa75-c527-429b-9acd-37b1b1f9c6c7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9whj6" Jul 28 17:10:52.992954 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:52.992551 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 17:10:52.992954 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:52.992633 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-networking-console-plugin-cert podName:8cc0aa75-c527-429b-9acd-37b1b1f9c6c7 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:54.992611899 +0000 UTC m=+39.721113227 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9whj6" (UID: "8cc0aa75-c527-429b-9acd-37b1b1f9c6c7") : secret "networking-console-plugin-cert" not found Jul 28 17:10:53.035548 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:53.035512 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xv5f2" event={"ID":"f16ab6fd-caec-4bb9-8aa8-d146b004e239","Type":"ContainerStarted","Data":"119419e9f996b8fcb5642a67951758c92e1c905b2b280272d4c6ddec7a327b35"} Jul 28 17:10:53.038818 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:53.038779 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-hzlqt" event={"ID":"5a580271-8c91-43b9-8479-21fedddcf96f","Type":"ContainerStarted","Data":"08a7a5197c4f1dca28e40a4204d4757b71a5ffa2cfa5bb602f938c9999442ebb"} Jul 28 17:10:53.043015 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:53.042983 2550 generic.go:356] "Generic (PLEG): container finished" podID="d9ae751d-09fa-4166-b47b-0e069281d796" containerID="2bdca2dc3d8c50488a85b88bfb242b2647629d3d498fa8dfba6495c1996e6311" exitCode=0 Jul 28 17:10:53.043159 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:53.043081 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qslrz" event={"ID":"d9ae751d-09fa-4166-b47b-0e069281d796","Type":"ContainerDied","Data":"2bdca2dc3d8c50488a85b88bfb242b2647629d3d498fa8dfba6495c1996e6311"} Jul 28 17:10:53.044925 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:53.044899 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp" event={"ID":"73e0baae-ba8a-4ef3-aff6-a31782400af6","Type":"ContainerStarted","Data":"11afdcaaae43dbb41097e9bdabf63a5f2087cf6efd7a92530979518c1126472f"} Jul 28 17:10:53.046414 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:53.046372 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fxdhd" event={"ID":"6f52b7f0-cbf9-46aa-bac6-b7fd7486ea50","Type":"ContainerStarted","Data":"630a589fc43bfa5cf9f4f5ee78fe47dcbce4237b78e1446716c101747555ad2a"} Jul 28 17:10:53.093820 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:53.093786 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b6d4302-fb6f-44e7-8b39-07a78c35b277-metrics-tls\") pod \"dns-default-jx7cl\" (UID: \"4b6d4302-fb6f-44e7-8b39-07a78c35b277\") " pod="openshift-dns/dns-default-jx7cl" Jul 28 17:10:53.094050 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:53.094029 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:10:53.094137 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:53.094123 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b6d4302-fb6f-44e7-8b39-07a78c35b277-metrics-tls podName:4b6d4302-fb6f-44e7-8b39-07a78c35b277 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:55.094100484 +0000 UTC m=+39.822601800 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b6d4302-fb6f-44e7-8b39-07a78c35b277-metrics-tls") pod "dns-default-jx7cl" (UID: "4b6d4302-fb6f-44e7-8b39-07a78c35b277") : secret "dns-default-metrics-tls" not found Jul 28 17:10:53.902539 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:53.902499 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-registry-tls\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:53.902742 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:53.902682 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:10:53.902742 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:53.902696 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5547bf85cb-jklcq: secret "image-registry-tls" not found Jul 28 17:10:53.902858 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:53.902759 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-registry-tls podName:39309997-adcd-4fb1-94d4-ac4668989199 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:55.902737083 +0000 UTC m=+40.631238408 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-registry-tls") pod "image-registry-5547bf85cb-jklcq" (UID: "39309997-adcd-4fb1-94d4-ac4668989199") : secret "image-registry-tls" not found Jul 28 17:10:54.054274 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:54.054235 2550 generic.go:356] "Generic (PLEG): container finished" podID="d9ae751d-09fa-4166-b47b-0e069281d796" containerID="59fe252f0803fe5d5def56e19957321de91cb0c927296d7998ddd4771c274947" exitCode=0 Jul 28 17:10:54.054741 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:54.054325 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qslrz" event={"ID":"d9ae751d-09fa-4166-b47b-0e069281d796","Type":"ContainerDied","Data":"59fe252f0803fe5d5def56e19957321de91cb0c927296d7998ddd4771c274947"} Jul 28 17:10:55.011497 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:55.011409 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9whj6\" (UID: \"8cc0aa75-c527-429b-9acd-37b1b1f9c6c7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9whj6" Jul 28 17:10:55.011633 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:55.011576 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 17:10:55.011670 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:55.011641 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-networking-console-plugin-cert podName:8cc0aa75-c527-429b-9acd-37b1b1f9c6c7 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:59.011622099 +0000 UTC m=+43.740123413 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9whj6" (UID: "8cc0aa75-c527-429b-9acd-37b1b1f9c6c7") : secret "networking-console-plugin-cert" not found Jul 28 17:10:55.112643 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:55.112592 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b6d4302-fb6f-44e7-8b39-07a78c35b277-metrics-tls\") pod \"dns-default-jx7cl\" (UID: \"4b6d4302-fb6f-44e7-8b39-07a78c35b277\") " pod="openshift-dns/dns-default-jx7cl" Jul 28 17:10:55.113122 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:55.112809 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:10:55.113122 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:55.112889 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b6d4302-fb6f-44e7-8b39-07a78c35b277-metrics-tls podName:4b6d4302-fb6f-44e7-8b39-07a78c35b277 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:59.112866143 +0000 UTC m=+43.841367473 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b6d4302-fb6f-44e7-8b39-07a78c35b277-metrics-tls") pod "dns-default-jx7cl" (UID: "4b6d4302-fb6f-44e7-8b39-07a78c35b277") : secret "dns-default-metrics-tls" not found Jul 28 17:10:55.921471 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:55.921244 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-registry-tls\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:55.921640 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:55.921426 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:10:55.921640 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:55.921569 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5547bf85cb-jklcq: secret "image-registry-tls" not found Jul 28 17:10:55.921743 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:55.921649 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-registry-tls podName:39309997-adcd-4fb1-94d4-ac4668989199 nodeName:}" failed. No retries permitted until 2026-07-28 17:10:59.921631178 +0000 UTC m=+44.650132515 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-registry-tls") pod "image-registry-5547bf85cb-jklcq" (UID: "39309997-adcd-4fb1-94d4-ac4668989199") : secret "image-registry-tls" not found Jul 28 17:10:57.066920 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:57.066876 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-qslrz" event={"ID":"d9ae751d-09fa-4166-b47b-0e069281d796","Type":"ContainerStarted","Data":"04214aef022552a2a074381c10b8604cedf7a06589df44cbf09c89e2b6453463"} Jul 28 17:10:57.109282 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:57.109229 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-qslrz" podStartSLOduration=10.804307400999999 podStartE2EDuration="41.109212805s" totalImagesPullingTime="30.304905404s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:57.105918501 +0000 UTC m=+41.834419838" watchObservedRunningTime="2026-07-28 17:10:57.109212805 +0000 UTC m=+41.837714142" Jul 28 17:10:58.071226 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:58.071184 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xv5f2" event={"ID":"f16ab6fd-caec-4bb9-8aa8-d146b004e239","Type":"ContainerStarted","Data":"3aea79587f2371c61120a178d1ab0fac3bb7c7029521dab66b06ed8520624c24"} Jul 28 17:10:58.072855 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:58.072827 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-hzlqt" event={"ID":"5a580271-8c91-43b9-8479-21fedddcf96f","Type":"ContainerStarted","Data":"65b6b9842b138caeedafe237052430c69d55ebab62162e33a6ff43c28b2c7443"} Jul 28 17:10:58.074450 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:58.074420 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp" event={"ID":"73e0baae-ba8a-4ef3-aff6-a31782400af6","Type":"ContainerStarted","Data":"0b6575162f168c20c90e2e799f6cd9aaf2b638e7177ae93c07603a0ac39353f1"} Jul 28 17:10:58.075885 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:58.075855 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fxdhd" event={"ID":"6f52b7f0-cbf9-46aa-bac6-b7fd7486ea50","Type":"ContainerStarted","Data":"d91718f216471c5aee3364964ef3b97f59ae426894ae9e9ae162289a602f79d4"} Jul 28 17:10:58.090407 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:58.090345 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xv5f2" podStartSLOduration=32.611246382 podStartE2EDuration="37.090331179s" totalImagesPullingTime="4.479084797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:10:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:58.090026091 +0000 UTC m=+42.818527429" watchObservedRunningTime="2026-07-28 17:10:58.090331179 +0000 UTC m=+42.818832545" Jul 28 17:10:58.108178 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:58.108127 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-hzlqt" podStartSLOduration=31.683737822 podStartE2EDuration="37.108109644s" totalImagesPullingTime="5.424371822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:10:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:58.106946479 +0000 UTC m=+42.835447813" watchObservedRunningTime="2026-07-28 17:10:58.108109644 +0000 UTC m=+42.836610981" Jul 28 17:10:58.127147 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:58.127093 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp" podStartSLOduration=34.676898379 podStartE2EDuration="39.127080797s" totalImagesPullingTime="4.450182418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:10:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:58.126282477 +0000 UTC m=+42.854783814" watchObservedRunningTime="2026-07-28 17:10:58.127080797 +0000 UTC m=+42.855582135" Jul 28 17:10:58.149243 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:58.149192 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fxdhd" podStartSLOduration=2.690874629 podStartE2EDuration="7.149173745s" totalImagesPullingTime="4.458299116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:10:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:10:58.148695734 +0000 UTC m=+42.877197065" watchObservedRunningTime="2026-07-28 17:10:58.149173745 +0000 UTC m=+42.877675085" Jul 28 17:10:59.049605 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:59.049546 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9whj6\" (UID: \"8cc0aa75-c527-429b-9acd-37b1b1f9c6c7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9whj6" Jul 28 17:10:59.049793 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:59.049690 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 17:10:59.049793 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:59.049763 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-networking-console-plugin-cert podName:8cc0aa75-c527-429b-9acd-37b1b1f9c6c7 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:07.049746662 +0000 UTC m=+51.778247976 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9whj6" (UID: "8cc0aa75-c527-429b-9acd-37b1b1f9c6c7") : secret "networking-console-plugin-cert" not found Jul 28 17:10:59.150940 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:59.150892 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b6d4302-fb6f-44e7-8b39-07a78c35b277-metrics-tls\") pod \"dns-default-jx7cl\" (UID: \"4b6d4302-fb6f-44e7-8b39-07a78c35b277\") " pod="openshift-dns/dns-default-jx7cl" Jul 28 17:10:59.151437 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:59.151115 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:10:59.151437 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:59.151203 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b6d4302-fb6f-44e7-8b39-07a78c35b277-metrics-tls podName:4b6d4302-fb6f-44e7-8b39-07a78c35b277 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:07.151176979 +0000 UTC m=+51.879678296 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b6d4302-fb6f-44e7-8b39-07a78c35b277-metrics-tls") pod "dns-default-jx7cl" (UID: "4b6d4302-fb6f-44e7-8b39-07a78c35b277") : secret "dns-default-metrics-tls" not found Jul 28 17:10:59.958211 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:10:59.958173 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-registry-tls\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:10:59.958459 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:59.958333 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:10:59.958459 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:59.958354 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5547bf85cb-jklcq: secret "image-registry-tls" not found Jul 28 17:10:59.958459 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:10:59.958450 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-registry-tls podName:39309997-adcd-4fb1-94d4-ac4668989199 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:07.958431621 +0000 UTC m=+52.686932970 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-registry-tls") pod "image-registry-5547bf85cb-jklcq" (UID: "39309997-adcd-4fb1-94d4-ac4668989199") : secret "image-registry-tls" not found Jul 28 17:11:07.111774 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:07.111733 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9whj6\" (UID: \"8cc0aa75-c527-429b-9acd-37b1b1f9c6c7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9whj6" Jul 28 17:11:07.112276 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:07.111910 2550 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 28 17:11:07.112276 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:07.111998 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-networking-console-plugin-cert podName:8cc0aa75-c527-429b-9acd-37b1b1f9c6c7 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:23.111975043 +0000 UTC m=+67.840476357 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-9whj6" (UID: "8cc0aa75-c527-429b-9acd-37b1b1f9c6c7") : secret "networking-console-plugin-cert" not found Jul 28 17:11:07.212354 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:07.212314 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b6d4302-fb6f-44e7-8b39-07a78c35b277-metrics-tls\") pod \"dns-default-jx7cl\" (UID: \"4b6d4302-fb6f-44e7-8b39-07a78c35b277\") " pod="openshift-dns/dns-default-jx7cl" Jul 28 17:11:07.212534 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:07.212487 2550 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 28 17:11:07.212571 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:07.212563 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b6d4302-fb6f-44e7-8b39-07a78c35b277-metrics-tls podName:4b6d4302-fb6f-44e7-8b39-07a78c35b277 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:23.212547164 +0000 UTC m=+67.941048481 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4b6d4302-fb6f-44e7-8b39-07a78c35b277-metrics-tls") pod "dns-default-jx7cl" (UID: "4b6d4302-fb6f-44e7-8b39-07a78c35b277") : secret "dns-default-metrics-tls" not found Jul 28 17:11:08.018945 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:08.018904 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-registry-tls\") pod \"image-registry-5547bf85cb-jklcq\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:11:08.019113 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:08.019036 2550 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 28 17:11:08.019113 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:08.019048 2550 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5547bf85cb-jklcq: secret "image-registry-tls" not found Jul 28 17:11:08.019113 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:08.019096 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-registry-tls podName:39309997-adcd-4fb1-94d4-ac4668989199 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:24.019083685 +0000 UTC m=+68.747585003 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-registry-tls") pod "image-registry-5547bf85cb-jklcq" (UID: "39309997-adcd-4fb1-94d4-ac4668989199") : secret "image-registry-tls" not found Jul 28 17:11:17.024258 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:17.024230 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4p62f" Jul 28 17:11:21.464246 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.464209 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-ggldr"] Jul 28 17:11:21.466988 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.466965 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ggldr" Jul 28 17:11:21.469842 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.469818 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8"] Jul 28 17:11:21.471612 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.471589 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-qx87z\"" Jul 28 17:11:21.471731 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.471642 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 28 17:11:21.471731 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.471661 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" Jul 28 17:11:21.472958 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.472933 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 28 17:11:21.473076 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.473004 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 28 17:11:21.473076 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.473020 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 28 17:11:21.474861 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.474837 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 28 17:11:21.474947 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.474902 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 28 17:11:21.475188 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.475169 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 28 17:11:21.475756 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.475741 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 28 17:11:21.476207 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.476192 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 28 17:11:21.476319 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.476304 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 28 17:11:21.476619 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.476600 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 28 17:11:21.517327 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.517290 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/71cf3148-1299-45dc-b42c-69508ad4cbf6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-695c8b497d-k2pq8\" (UID: \"71cf3148-1299-45dc-b42c-69508ad4cbf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" Jul 28 17:11:21.517327 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.517326 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f953c640-f9e2-4f72-971f-4719d48a19ef-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-ggldr\" (UID: \"f953c640-f9e2-4f72-971f-4719d48a19ef\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ggldr" Jul 28 17:11:21.517561 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.517353 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlln6\" (UniqueName: \"kubernetes.io/projected/f953c640-f9e2-4f72-971f-4719d48a19ef-kube-api-access-vlln6\") pod \"cluster-monitoring-operator-548b489cb8-ggldr\" (UID: \"f953c640-f9e2-4f72-971f-4719d48a19ef\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ggldr" Jul 28 17:11:21.517561 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.517440 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret\") pod \"global-pull-secret-syncer-gzjsf\" (UID: \"143142e3-d09f-4dde-bf59-1fffafc3e2fc\") " pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:11:21.517561 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.517463 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f953c640-f9e2-4f72-971f-4719d48a19ef-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ggldr\" (UID: \"f953c640-f9e2-4f72-971f-4719d48a19ef\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ggldr" Jul 28 17:11:21.517561 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.517519 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/71cf3148-1299-45dc-b42c-69508ad4cbf6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-695c8b497d-k2pq8\" (UID: \"71cf3148-1299-45dc-b42c-69508ad4cbf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" Jul 28 17:11:21.517561 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.517543 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-828lx\" (UniqueName: \"kubernetes.io/projected/71cf3148-1299-45dc-b42c-69508ad4cbf6-kube-api-access-828lx\") pod \"cluster-proxy-proxy-agent-695c8b497d-k2pq8\" (UID: \"71cf3148-1299-45dc-b42c-69508ad4cbf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" Jul 28 17:11:21.517793 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.517634 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/71cf3148-1299-45dc-b42c-69508ad4cbf6-ca\") pod \"cluster-proxy-proxy-agent-695c8b497d-k2pq8\" (UID: \"71cf3148-1299-45dc-b42c-69508ad4cbf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" Jul 28 17:11:21.517793 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.517668 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/71cf3148-1299-45dc-b42c-69508ad4cbf6-hub\") pod \"cluster-proxy-proxy-agent-695c8b497d-k2pq8\" (UID: \"71cf3148-1299-45dc-b42c-69508ad4cbf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" Jul 28 17:11:21.517793 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.517696 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/71cf3148-1299-45dc-b42c-69508ad4cbf6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-695c8b497d-k2pq8\" (UID: \"71cf3148-1299-45dc-b42c-69508ad4cbf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" Jul 28 17:11:21.517793 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.517733 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs\") pod \"network-metrics-daemon-4l6cw\" (UID: \"6cb7d969-3477-425b-9bf1-3192c9d7d94c\") " pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:11:21.518615 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.518595 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8"] Jul 28 17:11:21.519432 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.519414 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-ggldr"] Jul 28 17:11:21.521525 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.521510 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 28 17:11:21.521748 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.521731 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 28 17:11:21.531346 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.531323 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6cb7d969-3477-425b-9bf1-3192c9d7d94c-metrics-certs\") pod \"network-metrics-daemon-4l6cw\" (UID: \"6cb7d969-3477-425b-9bf1-3192c9d7d94c\") " pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:11:21.531475 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.531402 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/143142e3-d09f-4dde-bf59-1fffafc3e2fc-original-pull-secret\") pod \"global-pull-secret-syncer-gzjsf\" (UID: \"143142e3-d09f-4dde-bf59-1fffafc3e2fc\") " pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:11:21.579756 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.579712 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gzjsf" Jul 28 17:11:21.596212 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.596184 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xf5sm\"" Jul 28 17:11:21.603851 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.603817 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4l6cw" Jul 28 17:11:21.618729 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.618699 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/71cf3148-1299-45dc-b42c-69508ad4cbf6-ca\") pod \"cluster-proxy-proxy-agent-695c8b497d-k2pq8\" (UID: \"71cf3148-1299-45dc-b42c-69508ad4cbf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" Jul 28 17:11:21.618729 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.618732 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/71cf3148-1299-45dc-b42c-69508ad4cbf6-hub\") pod \"cluster-proxy-proxy-agent-695c8b497d-k2pq8\" (UID: \"71cf3148-1299-45dc-b42c-69508ad4cbf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" Jul 28 17:11:21.618962 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.618752 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/71cf3148-1299-45dc-b42c-69508ad4cbf6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-695c8b497d-k2pq8\" (UID: \"71cf3148-1299-45dc-b42c-69508ad4cbf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" Jul 28 17:11:21.618962 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.618798 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/71cf3148-1299-45dc-b42c-69508ad4cbf6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-695c8b497d-k2pq8\" (UID: \"71cf3148-1299-45dc-b42c-69508ad4cbf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" Jul 28 17:11:21.618962 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.618814 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f953c640-f9e2-4f72-971f-4719d48a19ef-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-ggldr\" (UID: \"f953c640-f9e2-4f72-971f-4719d48a19ef\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ggldr" Jul 28 17:11:21.618962 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.618836 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlln6\" (UniqueName: \"kubernetes.io/projected/f953c640-f9e2-4f72-971f-4719d48a19ef-kube-api-access-vlln6\") pod \"cluster-monitoring-operator-548b489cb8-ggldr\" (UID: \"f953c640-f9e2-4f72-971f-4719d48a19ef\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ggldr" Jul 28 17:11:21.618962 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.618855 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzr4x\" (UniqueName: \"kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x\") pod \"network-check-target-kh2q2\" (UID: \"50d07fcb-69de-4546-a128-9549657dd856\") " pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:11:21.618962 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.618882 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f953c640-f9e2-4f72-971f-4719d48a19ef-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ggldr\" (UID: \"f953c640-f9e2-4f72-971f-4719d48a19ef\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ggldr" Jul 28 17:11:21.618962 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.618920 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/71cf3148-1299-45dc-b42c-69508ad4cbf6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-695c8b497d-k2pq8\" (UID: \"71cf3148-1299-45dc-b42c-69508ad4cbf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" Jul 28 17:11:21.618962 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.618937 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-828lx\" (UniqueName: \"kubernetes.io/projected/71cf3148-1299-45dc-b42c-69508ad4cbf6-kube-api-access-828lx\") pod \"cluster-proxy-proxy-agent-695c8b497d-k2pq8\" (UID: \"71cf3148-1299-45dc-b42c-69508ad4cbf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" Jul 28 17:11:21.620164 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.620135 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/71cf3148-1299-45dc-b42c-69508ad4cbf6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-695c8b497d-k2pq8\" (UID: \"71cf3148-1299-45dc-b42c-69508ad4cbf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" Jul 28 17:11:21.620333 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.620204 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/f953c640-f9e2-4f72-971f-4719d48a19ef-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-ggldr\" (UID: \"f953c640-f9e2-4f72-971f-4719d48a19ef\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ggldr" Jul 28 17:11:21.622251 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.622200 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/71cf3148-1299-45dc-b42c-69508ad4cbf6-ca\") pod \"cluster-proxy-proxy-agent-695c8b497d-k2pq8\" (UID: \"71cf3148-1299-45dc-b42c-69508ad4cbf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" Jul 28 17:11:21.622378 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.622330 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/71cf3148-1299-45dc-b42c-69508ad4cbf6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-695c8b497d-k2pq8\" (UID: \"71cf3148-1299-45dc-b42c-69508ad4cbf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" Jul 28 17:11:21.623271 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.623249 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/71cf3148-1299-45dc-b42c-69508ad4cbf6-hub\") pod \"cluster-proxy-proxy-agent-695c8b497d-k2pq8\" (UID: \"71cf3148-1299-45dc-b42c-69508ad4cbf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" Jul 28 17:11:21.624023 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.623996 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/71cf3148-1299-45dc-b42c-69508ad4cbf6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-695c8b497d-k2pq8\" (UID: \"71cf3148-1299-45dc-b42c-69508ad4cbf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" Jul 28 17:11:21.624157 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.624139 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzr4x\" (UniqueName: \"kubernetes.io/projected/50d07fcb-69de-4546-a128-9549657dd856-kube-api-access-tzr4x\") pod \"network-check-target-kh2q2\" (UID: \"50d07fcb-69de-4546-a128-9549657dd856\") " pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:11:21.624470 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.624454 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/f953c640-f9e2-4f72-971f-4719d48a19ef-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-ggldr\" (UID: \"f953c640-f9e2-4f72-971f-4719d48a19ef\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ggldr" Jul 28 17:11:21.666494 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.666430 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlln6\" (UniqueName: \"kubernetes.io/projected/f953c640-f9e2-4f72-971f-4719d48a19ef-kube-api-access-vlln6\") pod \"cluster-monitoring-operator-548b489cb8-ggldr\" (UID: \"f953c640-f9e2-4f72-971f-4719d48a19ef\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ggldr" Jul 28 17:11:21.690489 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.690460 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-828lx\" (UniqueName: \"kubernetes.io/projected/71cf3148-1299-45dc-b42c-69508ad4cbf6-kube-api-access-828lx\") pod \"cluster-proxy-proxy-agent-695c8b497d-k2pq8\" (UID: \"71cf3148-1299-45dc-b42c-69508ad4cbf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" Jul 28 17:11:21.721136 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.721046 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5547bf85cb-jklcq"] Jul 28 17:11:21.721307 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:21.721285 2550 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" podUID="39309997-adcd-4fb1-94d4-ac4668989199" Jul 28 17:11:21.748343 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.748307 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gzjsf"] Jul 28 17:11:21.752047 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:11:21.752020 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod143142e3_d09f_4dde_bf59_1fffafc3e2fc.slice/crio-4ca1f0ee8b50c76d5b121a84222d8f42cb6b8f1c56c9bf0c886465a60d1b428b WatchSource:0}: Error finding container 4ca1f0ee8b50c76d5b121a84222d8f42cb6b8f1c56c9bf0c886465a60d1b428b: Status 404 returned error can't find the container with id 4ca1f0ee8b50c76d5b121a84222d8f42cb6b8f1c56c9bf0c886465a60d1b428b Jul 28 17:11:21.773302 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.773269 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4l6cw"] Jul 28 17:11:21.776279 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:11:21.776255 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cb7d969_3477_425b_9bf1_3192c9d7d94c.slice/crio-57a02972e210a1e691f380415ff4e870b2b712e6cdce51c09f3684749d5f37e6 WatchSource:0}: Error finding container 57a02972e210a1e691f380415ff4e870b2b712e6cdce51c09f3684749d5f37e6: Status 404 returned error can't find the container with id 57a02972e210a1e691f380415ff4e870b2b712e6cdce51c09f3684749d5f37e6 Jul 28 17:11:21.776995 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.776977 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ggldr" Jul 28 17:11:21.793841 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.793818 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" Jul 28 17:11:21.891481 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.891454 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wz9mb\"" Jul 28 17:11:21.898168 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.898084 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:11:21.944303 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.944260 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-ggldr"] Jul 28 17:11:21.947111 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:21.947048 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8"] Jul 28 17:11:21.949131 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:11:21.948655 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf953c640_f9e2_4f72_971f_4719d48a19ef.slice/crio-f38f2f158e940112507797b6a0a5ebfdc3820b62f1c3cb941132f2d4f73c8ff1 WatchSource:0}: Error finding container f38f2f158e940112507797b6a0a5ebfdc3820b62f1c3cb941132f2d4f73c8ff1: Status 404 returned error can't find the container with id f38f2f158e940112507797b6a0a5ebfdc3820b62f1c3cb941132f2d4f73c8ff1 Jul 28 17:11:21.949988 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:11:21.949947 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71cf3148_1299_45dc_b42c_69508ad4cbf6.slice/crio-05d27dd8d193041ba014dafd0c7f05ec9135be1ffee9a2cfa31921881d94ff14 WatchSource:0}: Error finding container 05d27dd8d193041ba014dafd0c7f05ec9135be1ffee9a2cfa31921881d94ff14: Status 404 returned error can't find the container with id 05d27dd8d193041ba014dafd0c7f05ec9135be1ffee9a2cfa31921881d94ff14 Jul 28 17:11:22.046546 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.046513 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kh2q2"] Jul 28 17:11:22.049507 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:11:22.049465 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50d07fcb_69de_4546_a128_9549657dd856.slice/crio-2396838294f4bd99f0196daae8199106b5a0dbd66f76d193a8379733cac1c37d WatchSource:0}: Error finding container 2396838294f4bd99f0196daae8199106b5a0dbd66f76d193a8379733cac1c37d: Status 404 returned error can't find the container with id 2396838294f4bd99f0196daae8199106b5a0dbd66f76d193a8379733cac1c37d Jul 28 17:11:22.137579 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.137543 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" event={"ID":"71cf3148-1299-45dc-b42c-69508ad4cbf6","Type":"ContainerStarted","Data":"05d27dd8d193041ba014dafd0c7f05ec9135be1ffee9a2cfa31921881d94ff14"} Jul 28 17:11:22.138955 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.138927 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kh2q2" event={"ID":"50d07fcb-69de-4546-a128-9549657dd856","Type":"ContainerStarted","Data":"5e947e75781b33294c88ad6336c88742317d924037269080f3aada499eb877eb"} Jul 28 17:11:22.139091 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.138979 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kh2q2" event={"ID":"50d07fcb-69de-4546-a128-9549657dd856","Type":"ContainerStarted","Data":"2396838294f4bd99f0196daae8199106b5a0dbd66f76d193a8379733cac1c37d"} Jul 28 17:11:22.139091 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.139046 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:11:22.140159 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.140135 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ggldr" event={"ID":"f953c640-f9e2-4f72-971f-4719d48a19ef","Type":"ContainerStarted","Data":"f38f2f158e940112507797b6a0a5ebfdc3820b62f1c3cb941132f2d4f73c8ff1"} Jul 28 17:11:22.141275 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.141254 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4l6cw" event={"ID":"6cb7d969-3477-425b-9bf1-3192c9d7d94c","Type":"ContainerStarted","Data":"57a02972e210a1e691f380415ff4e870b2b712e6cdce51c09f3684749d5f37e6"} Jul 28 17:11:22.142341 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.142327 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:11:22.142341 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.142333 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gzjsf" event={"ID":"143142e3-d09f-4dde-bf59-1fffafc3e2fc","Type":"ContainerStarted","Data":"4ca1f0ee8b50c76d5b121a84222d8f42cb6b8f1c56c9bf0c886465a60d1b428b"} Jul 28 17:11:22.148561 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.148540 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:11:22.184613 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.184545 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kh2q2" podStartSLOduration=66.184523617 podStartE2EDuration="1m6.184523617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:10:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:22.182737457 +0000 UTC m=+66.911238796" watchObservedRunningTime="2026-07-28 17:11:22.184523617 +0000 UTC m=+66.913024955" Jul 28 17:11:22.223822 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.223786 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39309997-adcd-4fb1-94d4-ac4668989199-registry-certificates\") pod \"39309997-adcd-4fb1-94d4-ac4668989199\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " Jul 28 17:11:22.223822 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.223827 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39309997-adcd-4fb1-94d4-ac4668989199-trusted-ca\") pod \"39309997-adcd-4fb1-94d4-ac4668989199\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " Jul 28 17:11:22.224088 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.223882 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39309997-adcd-4fb1-94d4-ac4668989199-ca-trust-extracted\") pod \"39309997-adcd-4fb1-94d4-ac4668989199\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " Jul 28 17:11:22.224088 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.223932 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bfpd\" (UniqueName: \"kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-kube-api-access-9bfpd\") pod \"39309997-adcd-4fb1-94d4-ac4668989199\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " Jul 28 17:11:22.224088 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.223971 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39309997-adcd-4fb1-94d4-ac4668989199-image-registry-private-configuration\") pod \"39309997-adcd-4fb1-94d4-ac4668989199\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " Jul 28 17:11:22.224088 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.223995 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-bound-sa-token\") pod \"39309997-adcd-4fb1-94d4-ac4668989199\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " Jul 28 17:11:22.224088 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.224019 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39309997-adcd-4fb1-94d4-ac4668989199-installation-pull-secrets\") pod \"39309997-adcd-4fb1-94d4-ac4668989199\" (UID: \"39309997-adcd-4fb1-94d4-ac4668989199\") " Jul 28 17:11:22.224313 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.224182 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39309997-adcd-4fb1-94d4-ac4668989199-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "39309997-adcd-4fb1-94d4-ac4668989199" (UID: "39309997-adcd-4fb1-94d4-ac4668989199"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:11:22.224313 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.224296 2550 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39309997-adcd-4fb1-94d4-ac4668989199-registry-certificates\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:11:22.224448 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.224306 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39309997-adcd-4fb1-94d4-ac4668989199-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "39309997-adcd-4fb1-94d4-ac4668989199" (UID: "39309997-adcd-4fb1-94d4-ac4668989199"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:11:22.224448 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.224342 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39309997-adcd-4fb1-94d4-ac4668989199-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "39309997-adcd-4fb1-94d4-ac4668989199" (UID: "39309997-adcd-4fb1-94d4-ac4668989199"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:11:22.226326 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.226295 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39309997-adcd-4fb1-94d4-ac4668989199-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "39309997-adcd-4fb1-94d4-ac4668989199" (UID: "39309997-adcd-4fb1-94d4-ac4668989199"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:11:22.226326 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.226305 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-kube-api-access-9bfpd" (OuterVolumeSpecName: "kube-api-access-9bfpd") pod "39309997-adcd-4fb1-94d4-ac4668989199" (UID: "39309997-adcd-4fb1-94d4-ac4668989199"). InnerVolumeSpecName "kube-api-access-9bfpd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:11:22.226620 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.226595 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "39309997-adcd-4fb1-94d4-ac4668989199" (UID: "39309997-adcd-4fb1-94d4-ac4668989199"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:11:22.226670 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.226595 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39309997-adcd-4fb1-94d4-ac4668989199-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "39309997-adcd-4fb1-94d4-ac4668989199" (UID: "39309997-adcd-4fb1-94d4-ac4668989199"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:11:22.325469 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.325436 2550 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39309997-adcd-4fb1-94d4-ac4668989199-image-registry-private-configuration\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:11:22.325469 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.325464 2550 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-bound-sa-token\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:11:22.325469 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.325474 2550 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39309997-adcd-4fb1-94d4-ac4668989199-installation-pull-secrets\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:11:22.325469 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.325484 2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39309997-adcd-4fb1-94d4-ac4668989199-trusted-ca\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:11:22.325741 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.325493 2550 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39309997-adcd-4fb1-94d4-ac4668989199-ca-trust-extracted\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:11:22.325741 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:22.325501 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bfpd\" (UniqueName: \"kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-kube-api-access-9bfpd\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:11:23.133750 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:23.133706 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9whj6\" (UID: \"8cc0aa75-c527-429b-9acd-37b1b1f9c6c7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9whj6" Jul 28 17:11:23.136877 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:23.136815 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8cc0aa75-c527-429b-9acd-37b1b1f9c6c7-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-9whj6\" (UID: \"8cc0aa75-c527-429b-9acd-37b1b1f9c6c7\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-9whj6" Jul 28 17:11:23.145609 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:23.145576 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5547bf85cb-jklcq" Jul 28 17:11:23.192237 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:23.192193 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5547bf85cb-jklcq"] Jul 28 17:11:23.195010 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:23.194979 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5547bf85cb-jklcq"] Jul 28 17:11:23.234287 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:23.234174 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b6d4302-fb6f-44e7-8b39-07a78c35b277-metrics-tls\") pod \"dns-default-jx7cl\" (UID: \"4b6d4302-fb6f-44e7-8b39-07a78c35b277\") " pod="openshift-dns/dns-default-jx7cl" Jul 28 17:11:23.234287 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:23.234259 2550 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39309997-adcd-4fb1-94d4-ac4668989199-registry-tls\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:11:23.237312 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:23.237288 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4b6d4302-fb6f-44e7-8b39-07a78c35b277-metrics-tls\") pod \"dns-default-jx7cl\" (UID: \"4b6d4302-fb6f-44e7-8b39-07a78c35b277\") " pod="openshift-dns/dns-default-jx7cl" Jul 28 17:11:23.373315 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:23.373113 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-t84jj\"" Jul 28 17:11:23.381097 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:23.381063 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9whj6" Jul 28 17:11:23.435354 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:23.435306 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hqvdh\"" Jul 28 17:11:23.442886 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:23.442607 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jx7cl" Jul 28 17:11:23.565022 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:23.564945 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-9whj6"] Jul 28 17:11:23.652192 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:23.652097 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jx7cl"] Jul 28 17:11:23.655071 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:11:23.655041 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b6d4302_fb6f_44e7_8b39_07a78c35b277.slice/crio-8861eea131dc3e2a980f8c0e15b3fbfc00628b811b6452fdcfa582a2a1000ec3 WatchSource:0}: Error finding container 8861eea131dc3e2a980f8c0e15b3fbfc00628b811b6452fdcfa582a2a1000ec3: Status 404 returned error can't find the container with id 8861eea131dc3e2a980f8c0e15b3fbfc00628b811b6452fdcfa582a2a1000ec3 Jul 28 17:11:23.772303 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:23.772261 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39309997-adcd-4fb1-94d4-ac4668989199" path="/var/lib/kubelet/pods/39309997-adcd-4fb1-94d4-ac4668989199/volumes" Jul 28 17:11:24.151839 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:24.151792 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4l6cw" event={"ID":"6cb7d969-3477-425b-9bf1-3192c9d7d94c","Type":"ContainerStarted","Data":"2ef6996f95c0891772006cbfa92840b3a964a00d51861e7ba151c00f66bb73f3"} Jul 28 17:11:24.152326 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:24.151856 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4l6cw" event={"ID":"6cb7d969-3477-425b-9bf1-3192c9d7d94c","Type":"ContainerStarted","Data":"f400378cc8f397e1aca385302f15382c170e58fc64954c8344e1ec33108b4616"} Jul 28 17:11:24.153566 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:24.153535 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jx7cl" event={"ID":"4b6d4302-fb6f-44e7-8b39-07a78c35b277","Type":"ContainerStarted","Data":"8861eea131dc3e2a980f8c0e15b3fbfc00628b811b6452fdcfa582a2a1000ec3"} Jul 28 17:11:24.154884 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:24.154847 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9whj6" event={"ID":"8cc0aa75-c527-429b-9acd-37b1b1f9c6c7","Type":"ContainerStarted","Data":"82fa1f63b73d922b425a57919120ddfffd41460009bf160c5a803a955b1770fb"} Jul 28 17:11:24.170182 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:24.170032 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4l6cw" podStartSLOduration=66.689605667 podStartE2EDuration="1m8.170012493s" totalImagesPullingTime="1.480406826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:10:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:24.169496944 +0000 UTC m=+68.897998281" watchObservedRunningTime="2026-07-28 17:11:24.170012493 +0000 UTC m=+68.898513832" Jul 28 17:11:24.958268 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:24.956979 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xg5vg"] Jul 28 17:11:24.977248 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:24.977197 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xg5vg"] Jul 28 17:11:24.977443 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:24.977329 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:24.980248 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:24.980220 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 28 17:11:24.981412 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:24.981335 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 28 17:11:24.981767 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:24.981621 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 28 17:11:24.981900 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:24.981887 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mtzhc\"" Jul 28 17:11:24.982315 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:24.982077 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 28 17:11:25.051234 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:25.051196 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-crio-socket\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:25.051425 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:25.051240 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztkwr\" (UniqueName: \"kubernetes.io/projected/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-kube-api-access-ztkwr\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:25.051425 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:25.051335 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:25.051425 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:25.051376 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-data-volume\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:25.051603 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:25.051467 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:25.152741 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:25.152702 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:25.153210 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:25.152766 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-crio-socket\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:25.153210 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:25.152795 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztkwr\" (UniqueName: \"kubernetes.io/projected/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-kube-api-access-ztkwr\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:25.153210 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:25.152839 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:25.153210 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:25.152883 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:11:25.153210 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:25.152911 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:11:25.153210 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:25.152924 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-crio-socket\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:25.153210 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:25.152966 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-insights-runtime-extractor-tls podName:8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:25.652942253 +0000 UTC m=+70.381443603 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xg5vg" (UID: "8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696") : secret "insights-runtime-extractor-tls" not found Jul 28 17:11:25.153210 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:25.152989 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-kube-rbac-proxy-cm podName:8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:25.652978019 +0000 UTC m=+70.381479340 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xg5vg" (UID: "8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696") : configmap "kube-rbac-proxy" not found Jul 28 17:11:25.153210 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:25.153010 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-data-volume\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:25.153665 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:25.153376 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-data-volume\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:25.165890 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:25.165855 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztkwr\" (UniqueName: \"kubernetes.io/projected/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-kube-api-access-ztkwr\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:25.659157 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:25.659044 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:25.659157 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:25.659127 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:25.659417 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:25.659202 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:11:25.659417 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:25.659255 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:11:25.659417 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:25.659303 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-kube-rbac-proxy-cm podName:8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:26.659278531 +0000 UTC m=+71.387779860 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xg5vg" (UID: "8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696") : configmap "kube-rbac-proxy" not found Jul 28 17:11:25.659417 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:25.659325 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-insights-runtime-extractor-tls podName:8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:26.659315218 +0000 UTC m=+71.387816534 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xg5vg" (UID: "8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696") : secret "insights-runtime-extractor-tls" not found Jul 28 17:11:26.668086 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:26.667999 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:26.668505 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:26.668087 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:26.668505 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:26.668157 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:11:26.668505 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:26.668185 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:11:26.668505 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:26.668226 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-insights-runtime-extractor-tls podName:8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:28.66820786 +0000 UTC m=+73.396709188 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xg5vg" (UID: "8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696") : secret "insights-runtime-extractor-tls" not found Jul 28 17:11:26.668505 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:26.668247 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-kube-rbac-proxy-cm podName:8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:28.668236303 +0000 UTC m=+73.396737620 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xg5vg" (UID: "8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696") : configmap "kube-rbac-proxy" not found Jul 28 17:11:27.131962 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:27.131930 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vpsrj_56e96be9-8647-407e-b2d9-f6718d894258/dns-node-resolver/0.log" Jul 28 17:11:27.167848 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:27.167809 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ggldr" event={"ID":"f953c640-f9e2-4f72-971f-4719d48a19ef","Type":"ContainerStarted","Data":"172206f1a43e242f3df72adfa906e45636fe532063b9252da6d815e4d9a0ae54"} Jul 28 17:11:27.730305 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:27.730275 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jblj2_c05afc7b-dee8-4de5-956e-336b79ac9867/node-ca/0.log" Jul 28 17:11:28.175999 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:28.175959 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gzjsf" event={"ID":"143142e3-d09f-4dde-bf59-1fffafc3e2fc","Type":"ContainerStarted","Data":"46c281e2d36bc5c8797423f2b110e674b8af6bb058a298c2a8f1f2c21c3432ec"} Jul 28 17:11:28.177377 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:28.177348 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jx7cl" event={"ID":"4b6d4302-fb6f-44e7-8b39-07a78c35b277","Type":"ContainerStarted","Data":"ce41ed5252d78c7d3f923f7ce3503ed2e77d2672e0e8e36fd639fd1134be68b9"} Jul 28 17:11:28.179659 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:28.179625 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9whj6" event={"ID":"8cc0aa75-c527-429b-9acd-37b1b1f9c6c7","Type":"ContainerStarted","Data":"4a4c9ebfbbba2d37e72569af928aea890d8cef68dfb1532d296f5a294a4b5afd"} Jul 28 17:11:28.198733 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:28.198686 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-ggldr" podStartSLOduration=2.842136189 podStartE2EDuration="7.198669096s" totalImagesPullingTime="4.356532907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:27.197708404 +0000 UTC m=+71.926209742" watchObservedRunningTime="2026-07-28 17:11:28.198669096 +0000 UTC m=+72.927170440" Jul 28 17:11:28.221488 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:28.221438 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gzjsf" podStartSLOduration=66.09953979 podStartE2EDuration="1m12.22142268s" totalImagesPullingTime="6.12188289s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:10:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:28.198258247 +0000 UTC m=+72.926759583" watchObservedRunningTime="2026-07-28 17:11:28.22142268 +0000 UTC m=+72.949924054" Jul 28 17:11:28.222323 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:28.222279 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-9whj6" podStartSLOduration=35.882261176 podStartE2EDuration="40.22226412s" totalImagesPullingTime="4.340002944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:10:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:28.220095226 +0000 UTC m=+72.948596562" watchObservedRunningTime="2026-07-28 17:11:28.22226412 +0000 UTC m=+72.950765460" Jul 28 17:11:28.689235 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:28.689175 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:28.689444 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:28.689257 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:28.689444 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:28.689340 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:11:28.689444 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:28.689341 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:11:28.689444 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:28.689435 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-kube-rbac-proxy-cm podName:8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:32.689417645 +0000 UTC m=+77.417918963 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xg5vg" (UID: "8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696") : configmap "kube-rbac-proxy" not found Jul 28 17:11:28.689593 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:28.689454 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-insights-runtime-extractor-tls podName:8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:32.689446896 +0000 UTC m=+77.417948209 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xg5vg" (UID: "8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696") : secret "insights-runtime-extractor-tls" not found Jul 28 17:11:28.730570 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:28.730541 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fxdhd_6f52b7f0-cbf9-46aa-bac6-b7fd7486ea50/serve-healthcheck-canary/0.log" Jul 28 17:11:29.186972 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:29.186933 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jx7cl" event={"ID":"4b6d4302-fb6f-44e7-8b39-07a78c35b277","Type":"ContainerStarted","Data":"acc007ec48679dec97b1182876a785c522aaec724eb85e529510632db30a3ec6"} Jul 28 17:11:29.207592 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:29.207542 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jx7cl" podStartSLOduration=33.947722047 podStartE2EDuration="38.207521228s" totalImagesPullingTime="4.259799181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:10:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:29.205419862 +0000 UTC m=+73.933921199" watchObservedRunningTime="2026-07-28 17:11:29.207521228 +0000 UTC m=+73.936022566" Jul 28 17:11:30.191680 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:30.191644 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jx7cl" Jul 28 17:11:32.200120 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:32.200080 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" event={"ID":"71cf3148-1299-45dc-b42c-69508ad4cbf6","Type":"ContainerStarted","Data":"a4910e84304a17a5c00e0bbef48c7cdca4083674d52cd70ac811f7afcff4ac95"} Jul 28 17:11:32.200120 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:32.200126 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" event={"ID":"71cf3148-1299-45dc-b42c-69508ad4cbf6","Type":"ContainerStarted","Data":"758b3483e7ff33884b2fb38cd701474657a3845d8dc33745b48c8bd47b31b188"} Jul 28 17:11:32.200624 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:32.200137 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" event={"ID":"71cf3148-1299-45dc-b42c-69508ad4cbf6","Type":"ContainerStarted","Data":"2b011699aef77f7ce4415bdda33a61b3aa7375419837230728865be226b74a57"} Jul 28 17:11:32.221504 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:32.221446 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-695c8b497d-k2pq8" podStartSLOduration=1.559882343 podStartE2EDuration="11.221431422s" totalImagesPullingTime="9.661549079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:32.219888 +0000 UTC m=+76.948389363" watchObservedRunningTime="2026-07-28 17:11:32.221431422 +0000 UTC m=+76.949932758" Jul 28 17:11:32.727187 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:32.727148 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:32.727360 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:32.727214 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:32.727360 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:32.727274 2550 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 28 17:11:32.727360 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:32.727332 2550 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 28 17:11:32.727360 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:32.727340 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-kube-rbac-proxy-cm podName:8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:40.727323381 +0000 UTC m=+85.455824695 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xg5vg" (UID: "8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696") : configmap "kube-rbac-proxy" not found Jul 28 17:11:32.727565 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:32.727410 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-insights-runtime-extractor-tls podName:8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696 nodeName:}" failed. No retries permitted until 2026-07-28 17:11:40.727368508 +0000 UTC m=+85.455869825 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xg5vg" (UID: "8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696") : secret "insights-runtime-extractor-tls" not found Jul 28 17:11:36.222402 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.222354 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78d99d45cc-czwxg"] Jul 28 17:11:36.225237 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.225221 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:36.228526 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.228501 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 28 17:11:36.228633 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.228564 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 28 17:11:36.229813 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.229797 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-6j5sw\"" Jul 28 17:11:36.230062 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.230038 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 28 17:11:36.230062 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.230049 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 28 17:11:36.230264 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.230079 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 28 17:11:36.230264 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.230155 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 28 17:11:36.230462 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.230380 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 28 17:11:36.239733 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.239711 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78d99d45cc-czwxg"] Jul 28 17:11:36.355420 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.355368 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnmds\" (UniqueName: \"kubernetes.io/projected/d814005a-8155-4b28-b03f-8ce036e19757-kube-api-access-pnmds\") pod \"console-78d99d45cc-czwxg\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:36.355420 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.355426 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d814005a-8155-4b28-b03f-8ce036e19757-console-config\") pod \"console-78d99d45cc-czwxg\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:36.355630 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.355477 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d814005a-8155-4b28-b03f-8ce036e19757-console-serving-cert\") pod \"console-78d99d45cc-czwxg\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:36.355630 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.355540 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d814005a-8155-4b28-b03f-8ce036e19757-console-oauth-config\") pod \"console-78d99d45cc-czwxg\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:36.355630 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.355589 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d814005a-8155-4b28-b03f-8ce036e19757-service-ca\") pod \"console-78d99d45cc-czwxg\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:36.355630 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.355620 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d814005a-8155-4b28-b03f-8ce036e19757-oauth-serving-cert\") pod \"console-78d99d45cc-czwxg\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:36.460687 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.460645 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d814005a-8155-4b28-b03f-8ce036e19757-service-ca\") pod \"console-78d99d45cc-czwxg\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:36.460848 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.460694 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d814005a-8155-4b28-b03f-8ce036e19757-oauth-serving-cert\") pod \"console-78d99d45cc-czwxg\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:36.460848 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.460773 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnmds\" (UniqueName: \"kubernetes.io/projected/d814005a-8155-4b28-b03f-8ce036e19757-kube-api-access-pnmds\") pod \"console-78d99d45cc-czwxg\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:36.460848 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.460799 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d814005a-8155-4b28-b03f-8ce036e19757-console-config\") pod \"console-78d99d45cc-czwxg\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:36.460848 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.460844 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d814005a-8155-4b28-b03f-8ce036e19757-console-serving-cert\") pod \"console-78d99d45cc-czwxg\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:36.461081 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.460866 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d814005a-8155-4b28-b03f-8ce036e19757-console-oauth-config\") pod \"console-78d99d45cc-czwxg\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:36.461519 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.461491 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d814005a-8155-4b28-b03f-8ce036e19757-console-config\") pod \"console-78d99d45cc-czwxg\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:36.461633 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.461532 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d814005a-8155-4b28-b03f-8ce036e19757-service-ca\") pod \"console-78d99d45cc-czwxg\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:36.461633 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.461536 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d814005a-8155-4b28-b03f-8ce036e19757-oauth-serving-cert\") pod \"console-78d99d45cc-czwxg\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:36.463372 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.463352 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d814005a-8155-4b28-b03f-8ce036e19757-console-oauth-config\") pod \"console-78d99d45cc-czwxg\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:36.463442 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.463356 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d814005a-8155-4b28-b03f-8ce036e19757-console-serving-cert\") pod \"console-78d99d45cc-czwxg\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:36.476974 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.476919 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnmds\" (UniqueName: \"kubernetes.io/projected/d814005a-8155-4b28-b03f-8ce036e19757-kube-api-access-pnmds\") pod \"console-78d99d45cc-czwxg\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:36.534823 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.534784 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:36.666519 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:36.666473 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78d99d45cc-czwxg"] Jul 28 17:11:36.668994 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:11:36.668961 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd814005a_8155_4b28_b03f_8ce036e19757.slice/crio-a64f858e37970c32beb760e962078857eabc3107889e4c919ffd7e9230fb32a7 WatchSource:0}: Error finding container a64f858e37970c32beb760e962078857eabc3107889e4c919ffd7e9230fb32a7: Status 404 returned error can't find the container with id a64f858e37970c32beb760e962078857eabc3107889e4c919ffd7e9230fb32a7 Jul 28 17:11:37.218467 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:37.218420 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78d99d45cc-czwxg" event={"ID":"d814005a-8155-4b28-b03f-8ce036e19757","Type":"ContainerStarted","Data":"a64f858e37970c32beb760e962078857eabc3107889e4c919ffd7e9230fb32a7"} Jul 28 17:11:39.366754 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.366721 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-kc7cf"] Jul 28 17:11:39.369851 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.369829 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.372421 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.372382 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 28 17:11:39.372535 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.372509 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 28 17:11:39.373471 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.373448 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 28 17:11:39.373581 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.373516 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 28 17:11:39.373930 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.373905 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-c44q6\"" Jul 28 17:11:39.485768 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.485737 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-node-exporter-wtmp\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.485937 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.485781 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-sys\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.485937 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.485806 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-node-exporter-textfile\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.485937 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.485896 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-node-exporter-tls\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.486072 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.485941 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-root\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.486072 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.485985 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztl6p\" (UniqueName: \"kubernetes.io/projected/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-kube-api-access-ztl6p\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.486072 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.486012 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-node-exporter-accelerators-collector-config\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.486072 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.486040 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.486199 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.486108 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-metrics-client-ca\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.587178 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.587099 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-root\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.587178 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.587141 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztl6p\" (UniqueName: \"kubernetes.io/projected/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-kube-api-access-ztl6p\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.587178 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.587162 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-node-exporter-accelerators-collector-config\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.587452 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.587183 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.587452 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.587231 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-metrics-client-ca\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.587452 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.587298 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-node-exporter-wtmp\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.587452 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.587344 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-sys\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.587452 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.587369 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-node-exporter-textfile\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.587452 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.587423 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-node-exporter-tls\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.587452 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.587442 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-node-exporter-wtmp\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.587794 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.587499 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-sys\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.587794 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.587230 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-root\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.587794 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:39.587539 2550 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 28 17:11:39.587794 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:11:39.587614 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-node-exporter-tls podName:e30b4440-4d5f-4bc1-91ed-b67bc5af1aad nodeName:}" failed. No retries permitted until 2026-07-28 17:11:40.087593693 +0000 UTC m=+84.816095042 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-node-exporter-tls") pod "node-exporter-kc7cf" (UID: "e30b4440-4d5f-4bc1-91ed-b67bc5af1aad") : secret "node-exporter-tls" not found Jul 28 17:11:39.587794 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.587792 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-node-exporter-textfile\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.588064 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.587906 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-metrics-client-ca\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.588064 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.587959 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-node-exporter-accelerators-collector-config\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.589552 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.589529 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:39.596860 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:39.596835 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztl6p\" (UniqueName: \"kubernetes.io/projected/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-kube-api-access-ztl6p\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:40.092639 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:40.092602 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-node-exporter-tls\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:40.094895 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:40.094869 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e30b4440-4d5f-4bc1-91ed-b67bc5af1aad-node-exporter-tls\") pod \"node-exporter-kc7cf\" (UID: \"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad\") " pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:40.197235 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:40.197204 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jx7cl" Jul 28 17:11:40.251037 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:40.250865 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78d99d45cc-czwxg" event={"ID":"d814005a-8155-4b28-b03f-8ce036e19757","Type":"ContainerStarted","Data":"91faed0def14316756885dfc08db33d0eeb896da99e04ee27f3187c9fbf22ca2"} Jul 28 17:11:40.277832 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:40.277744 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78d99d45cc-czwxg" podStartSLOduration=1.63511272 podStartE2EDuration="4.27772434s" totalImagesPullingTime="2.64261162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:40.277319051 +0000 UTC m=+85.005820379" watchObservedRunningTime="2026-07-28 17:11:40.27772434 +0000 UTC m=+85.006225676" Jul 28 17:11:40.279369 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:40.279347 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-kc7cf" Jul 28 17:11:40.292107 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:11:40.292076 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode30b4440_4d5f_4bc1_91ed_b67bc5af1aad.slice/crio-d706ed6f209a0f95cfef2cca68aa1e7675b090d83723db45543cf088e71aad51 WatchSource:0}: Error finding container d706ed6f209a0f95cfef2cca68aa1e7675b090d83723db45543cf088e71aad51: Status 404 returned error can't find the container with id d706ed6f209a0f95cfef2cca68aa1e7675b090d83723db45543cf088e71aad51 Jul 28 17:11:40.797897 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:40.797858 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:40.798418 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:40.797927 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:40.798499 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:40.798472 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:40.800664 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:40.800642 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xg5vg\" (UID: \"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696\") " pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:40.890760 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:40.890722 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xg5vg" Jul 28 17:11:41.038492 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:41.038439 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xg5vg"] Jul 28 17:11:41.141846 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:11:41.141760 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bce64c0_eeb2_43b9_aca7_ed3b7bbc7696.slice/crio-480e1a5583231d766d28508c0fde277cc9282ee5bf43365817047b6d9da0144d WatchSource:0}: Error finding container 480e1a5583231d766d28508c0fde277cc9282ee5bf43365817047b6d9da0144d: Status 404 returned error can't find the container with id 480e1a5583231d766d28508c0fde277cc9282ee5bf43365817047b6d9da0144d Jul 28 17:11:41.255480 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:41.255429 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xg5vg" event={"ID":"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696","Type":"ContainerStarted","Data":"480e1a5583231d766d28508c0fde277cc9282ee5bf43365817047b6d9da0144d"} Jul 28 17:11:41.256998 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:41.256964 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kc7cf" event={"ID":"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad","Type":"ContainerStarted","Data":"9e40197ce6b8a8fe4f357d9509de137cbf753657a09c306347114185d40f6a1d"} Jul 28 17:11:41.257134 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:41.257035 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kc7cf" event={"ID":"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad","Type":"ContainerStarted","Data":"d706ed6f209a0f95cfef2cca68aa1e7675b090d83723db45543cf088e71aad51"} Jul 28 17:11:42.264122 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.264080 2550 generic.go:356] "Generic (PLEG): container finished" podID="e30b4440-4d5f-4bc1-91ed-b67bc5af1aad" containerID="9e40197ce6b8a8fe4f357d9509de137cbf753657a09c306347114185d40f6a1d" exitCode=0 Jul 28 17:11:42.264637 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.264162 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kc7cf" event={"ID":"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad","Type":"ContainerDied","Data":"9e40197ce6b8a8fe4f357d9509de137cbf753657a09c306347114185d40f6a1d"} Jul 28 17:11:42.368456 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.368422 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-f67645548-v65s7"] Jul 28 17:11:42.373512 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.373482 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.376976 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.376953 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 28 17:11:42.377290 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.377186 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 28 17:11:42.377407 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.377308 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 28 17:11:42.377407 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.377325 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 28 17:11:42.377519 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.377438 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-eismb8hv5064t\"" Jul 28 17:11:42.377608 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.377590 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-j2rl2\"" Jul 28 17:11:42.377670 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.377595 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 28 17:11:42.398600 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.398568 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-f67645548-v65s7"] Jul 28 17:11:42.511758 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.511718 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e8c42df-5eea-48e7-bf00-15364556b04b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.512019 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.511769 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw5tv\" (UniqueName: \"kubernetes.io/projected/7e8c42df-5eea-48e7-bf00-15364556b04b-kube-api-access-sw5tv\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.512019 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.511796 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7e8c42df-5eea-48e7-bf00-15364556b04b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.512019 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.511844 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e8c42df-5eea-48e7-bf00-15364556b04b-secret-grpc-tls\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.512019 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.511884 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e8c42df-5eea-48e7-bf00-15364556b04b-metrics-client-ca\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.512019 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.511917 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e8c42df-5eea-48e7-bf00-15364556b04b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.512019 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.511965 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7e8c42df-5eea-48e7-bf00-15364556b04b-secret-thanos-querier-tls\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.512019 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.511994 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7e8c42df-5eea-48e7-bf00-15364556b04b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.613267 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.613214 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e8c42df-5eea-48e7-bf00-15364556b04b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.613502 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.613280 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw5tv\" (UniqueName: \"kubernetes.io/projected/7e8c42df-5eea-48e7-bf00-15364556b04b-kube-api-access-sw5tv\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.613502 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.613308 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7e8c42df-5eea-48e7-bf00-15364556b04b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.613502 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.613333 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e8c42df-5eea-48e7-bf00-15364556b04b-secret-grpc-tls\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.613502 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.613362 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e8c42df-5eea-48e7-bf00-15364556b04b-metrics-client-ca\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.613502 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.613401 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e8c42df-5eea-48e7-bf00-15364556b04b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.613502 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.613452 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7e8c42df-5eea-48e7-bf00-15364556b04b-secret-thanos-querier-tls\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.613502 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.613483 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7e8c42df-5eea-48e7-bf00-15364556b04b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.614974 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.614922 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e8c42df-5eea-48e7-bf00-15364556b04b-metrics-client-ca\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.616508 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.616461 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7e8c42df-5eea-48e7-bf00-15364556b04b-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.616508 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.616490 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/7e8c42df-5eea-48e7-bf00-15364556b04b-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.617008 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.616982 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7e8c42df-5eea-48e7-bf00-15364556b04b-secret-grpc-tls\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.617097 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.617066 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/7e8c42df-5eea-48e7-bf00-15364556b04b-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.617213 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.617192 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7e8c42df-5eea-48e7-bf00-15364556b04b-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.617260 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.617197 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/7e8c42df-5eea-48e7-bf00-15364556b04b-secret-thanos-querier-tls\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.624577 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.624549 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw5tv\" (UniqueName: \"kubernetes.io/projected/7e8c42df-5eea-48e7-bf00-15364556b04b-kube-api-access-sw5tv\") pod \"thanos-querier-f67645548-v65s7\" (UID: \"7e8c42df-5eea-48e7-bf00-15364556b04b\") " pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.689467 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.689425 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:42.871993 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:42.871958 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-f67645548-v65s7"] Jul 28 17:11:42.873815 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:11:42.873778 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e8c42df_5eea_48e7_bf00_15364556b04b.slice/crio-afcea4ba00623ad7917806a6c02268f7650a4c60ad44bc0f9f8cce275d16695f WatchSource:0}: Error finding container afcea4ba00623ad7917806a6c02268f7650a4c60ad44bc0f9f8cce275d16695f: Status 404 returned error can't find the container with id afcea4ba00623ad7917806a6c02268f7650a4c60ad44bc0f9f8cce275d16695f Jul 28 17:11:43.270117 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:43.270029 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kc7cf" event={"ID":"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad","Type":"ContainerStarted","Data":"d20ab19e97e71b8958932e86d34104dd1964ce5da0b640f7d70cf0c95a6c4314"} Jul 28 17:11:43.270117 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:43.270103 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-kc7cf" event={"ID":"e30b4440-4d5f-4bc1-91ed-b67bc5af1aad","Type":"ContainerStarted","Data":"1689e21512d7af60ad1fc4bd046db2f0281945173c86b6cc65a57d02ddad81f0"} Jul 28 17:11:43.271833 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:43.271802 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f67645548-v65s7" event={"ID":"7e8c42df-5eea-48e7-bf00-15364556b04b","Type":"ContainerStarted","Data":"afcea4ba00623ad7917806a6c02268f7650a4c60ad44bc0f9f8cce275d16695f"} Jul 28 17:11:43.296653 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:43.296019 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-kc7cf" podStartSLOduration=3.333135361 podStartE2EDuration="4.295995299s" totalImagesPullingTime="896.212707ms" totalInitContainerRuntime="66.647231ms" isStatefulPod=true podCreationTimestamp="2026-07-28 17:11:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:43.295229138 +0000 UTC m=+88.023730474" watchObservedRunningTime="2026-07-28 17:11:43.295995299 +0000 UTC m=+88.024496637" Jul 28 17:11:45.645478 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.645437 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6ccd4d6686-lkn2d"] Jul 28 17:11:45.649005 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.648977 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.658622 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.658600 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 28 17:11:45.666539 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.666473 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6ccd4d6686-lkn2d"] Jul 28 17:11:45.740281 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.740230 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-trusted-ca-bundle\") pod \"console-6ccd4d6686-lkn2d\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.740486 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.740324 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-console-config\") pod \"console-6ccd4d6686-lkn2d\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.740486 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.740424 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-service-ca\") pod \"console-6ccd4d6686-lkn2d\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.740486 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.740475 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f81f663-9aac-471e-8ac2-6e05ceaf5111-console-oauth-config\") pod \"console-6ccd4d6686-lkn2d\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.740629 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.740496 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f81f663-9aac-471e-8ac2-6e05ceaf5111-console-serving-cert\") pod \"console-6ccd4d6686-lkn2d\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.740629 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.740533 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-oauth-serving-cert\") pod \"console-6ccd4d6686-lkn2d\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.740629 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.740564 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vkrm\" (UniqueName: \"kubernetes.io/projected/1f81f663-9aac-471e-8ac2-6e05ceaf5111-kube-api-access-8vkrm\") pod \"console-6ccd4d6686-lkn2d\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.840998 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.840955 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-console-config\") pod \"console-6ccd4d6686-lkn2d\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.840998 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.841007 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-service-ca\") pod \"console-6ccd4d6686-lkn2d\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.841243 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.841050 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f81f663-9aac-471e-8ac2-6e05ceaf5111-console-oauth-config\") pod \"console-6ccd4d6686-lkn2d\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.841243 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.841081 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f81f663-9aac-471e-8ac2-6e05ceaf5111-console-serving-cert\") pod \"console-6ccd4d6686-lkn2d\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.841243 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.841222 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-oauth-serving-cert\") pod \"console-6ccd4d6686-lkn2d\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.841419 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.841273 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vkrm\" (UniqueName: \"kubernetes.io/projected/1f81f663-9aac-471e-8ac2-6e05ceaf5111-kube-api-access-8vkrm\") pod \"console-6ccd4d6686-lkn2d\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.841419 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.841314 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-trusted-ca-bundle\") pod \"console-6ccd4d6686-lkn2d\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.841852 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.841824 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-console-config\") pod \"console-6ccd4d6686-lkn2d\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.841973 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.841876 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-service-ca\") pod \"console-6ccd4d6686-lkn2d\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.842042 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.842005 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-oauth-serving-cert\") pod \"console-6ccd4d6686-lkn2d\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.842264 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.842240 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-trusted-ca-bundle\") pod \"console-6ccd4d6686-lkn2d\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.844533 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.844510 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f81f663-9aac-471e-8ac2-6e05ceaf5111-console-oauth-config\") pod \"console-6ccd4d6686-lkn2d\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.844696 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.844670 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f81f663-9aac-471e-8ac2-6e05ceaf5111-console-serving-cert\") pod \"console-6ccd4d6686-lkn2d\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.851069 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.851043 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vkrm\" (UniqueName: \"kubernetes.io/projected/1f81f663-9aac-471e-8ac2-6e05ceaf5111-kube-api-access-8vkrm\") pod \"console-6ccd4d6686-lkn2d\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:45.959768 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:45.959691 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:46.535545 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:46.535502 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:46.535742 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:46.535581 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:46.541602 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:46.541569 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:46.707808 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:46.707726 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6ccd4d6686-lkn2d"] Jul 28 17:11:46.710712 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:11:46.710683 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f81f663_9aac_471e_8ac2_6e05ceaf5111.slice/crio-9a3724d7096a2c1adbdbbc332db6e79c84a6b9ecd7b4bfa2a3a818d954e6f307 WatchSource:0}: Error finding container 9a3724d7096a2c1adbdbbc332db6e79c84a6b9ecd7b4bfa2a3a818d954e6f307: Status 404 returned error can't find the container with id 9a3724d7096a2c1adbdbbc332db6e79c84a6b9ecd7b4bfa2a3a818d954e6f307 Jul 28 17:11:47.290073 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:47.290030 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xg5vg" event={"ID":"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696","Type":"ContainerStarted","Data":"1b5cbeb156b4adcbb35cd6f8e03c6cbc182b9c64cb5e52302d30f8f732c14d5d"} Jul 28 17:11:47.292829 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:47.292739 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ccd4d6686-lkn2d" event={"ID":"1f81f663-9aac-471e-8ac2-6e05ceaf5111","Type":"ContainerStarted","Data":"ea5b26108f7f89158bd6c043e44ffaf3a742efca6b30f46f18d808e238614be2"} Jul 28 17:11:47.292829 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:47.292798 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ccd4d6686-lkn2d" event={"ID":"1f81f663-9aac-471e-8ac2-6e05ceaf5111","Type":"ContainerStarted","Data":"9a3724d7096a2c1adbdbbc332db6e79c84a6b9ecd7b4bfa2a3a818d954e6f307"} Jul 28 17:11:47.295612 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:47.295579 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f67645548-v65s7" event={"ID":"7e8c42df-5eea-48e7-bf00-15364556b04b","Type":"ContainerStarted","Data":"bcc7cc1ef1e622b4cb49939f7b6072600794d9f6abbf7d2c512334f8d1a9d559"} Jul 28 17:11:47.295743 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:47.295659 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f67645548-v65s7" event={"ID":"7e8c42df-5eea-48e7-bf00-15364556b04b","Type":"ContainerStarted","Data":"bc45e1fb24798d5c12a0027fc0e7b0feb3b580de96133a458478bca6c359728a"} Jul 28 17:11:47.295743 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:47.295701 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f67645548-v65s7" event={"ID":"7e8c42df-5eea-48e7-bf00-15364556b04b","Type":"ContainerStarted","Data":"41e57a227bc20720884c53146dccdf2e106effee1854bcd90f570ef484d037e4"} Jul 28 17:11:47.301160 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:47.301121 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:11:47.315686 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:47.315558 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6ccd4d6686-lkn2d" podStartSLOduration=2.31553543 podStartE2EDuration="2.31553543s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:47.314733566 +0000 UTC m=+92.043234901" watchObservedRunningTime="2026-07-28 17:11:47.31553543 +0000 UTC m=+92.044036767" Jul 28 17:11:48.304169 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:48.303994 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f67645548-v65s7" event={"ID":"7e8c42df-5eea-48e7-bf00-15364556b04b","Type":"ContainerStarted","Data":"9817e21aa793bd2325f38ff730ccd48aa61297f2be4f2f24ec0955c83dba25db"} Jul 28 17:11:48.304169 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:48.304075 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f67645548-v65s7" event={"ID":"7e8c42df-5eea-48e7-bf00-15364556b04b","Type":"ContainerStarted","Data":"71850fbb9da81b87ee62d6e976b2683353072bceb5aae676257471ad61544b53"} Jul 28 17:11:48.304169 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:48.304115 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-f67645548-v65s7" event={"ID":"7e8c42df-5eea-48e7-bf00-15364556b04b","Type":"ContainerStarted","Data":"b91dfd8e5e4be2a6248f02df6fbef91d807b22d375229cebe04a272fef130c96"} Jul 28 17:11:48.331403 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:48.331351 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-f67645548-v65s7" podStartSLOduration=1.508248725 podStartE2EDuration="6.331335099s" totalImagesPullingTime="4.823086374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:11:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:48.329526084 +0000 UTC m=+93.058027420" watchObservedRunningTime="2026-07-28 17:11:48.331335099 +0000 UTC m=+93.059836412" Jul 28 17:11:49.308687 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:49.308649 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:51.319255 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:51.319213 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xg5vg" event={"ID":"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696","Type":"ContainerStarted","Data":"b64ba094534a24359550fc40f04fbc37155e36e511d4009d2870040cf283b456"} Jul 28 17:11:53.148698 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:53.148659 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kh2q2" Jul 28 17:11:54.337122 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:54.337085 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xg5vg" event={"ID":"8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696","Type":"ContainerStarted","Data":"3e5ba99ce0e0cb0ad73a518a520109df99ffcc0615c1c36ac92ed30145747488"} Jul 28 17:11:54.357708 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:54.357649 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xg5vg" podStartSLOduration=17.688751905 podStartE2EDuration="30.357630451s" totalImagesPullingTime="12.668878546s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 17:11:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:11:54.356639151 +0000 UTC m=+99.085140487" watchObservedRunningTime="2026-07-28 17:11:54.357630451 +0000 UTC m=+99.086131786" Jul 28 17:11:55.318188 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:55.318156 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-f67645548-v65s7" Jul 28 17:11:55.960466 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:55.960425 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:55.960957 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:55.960504 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:55.964945 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:55.964920 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:56.347527 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:56.347493 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:11:56.410760 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:11:56.410727 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78d99d45cc-czwxg"] Jul 28 17:12:21.429868 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:21.429809 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78d99d45cc-czwxg" podUID="d814005a-8155-4b28-b03f-8ce036e19757" containerName="console" containerID="cri-o://91faed0def14316756885dfc08db33d0eeb896da99e04ee27f3187c9fbf22ca2" gracePeriod=15 Jul 28 17:12:22.210487 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.210451 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:12:22.355335 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.355245 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pnmds\" (UniqueName: \"kubernetes.io/projected/d814005a-8155-4b28-b03f-8ce036e19757-kube-api-access-pnmds\") pod \"d814005a-8155-4b28-b03f-8ce036e19757\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " Jul 28 17:12:22.355536 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.355340 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d814005a-8155-4b28-b03f-8ce036e19757-console-serving-cert\") pod \"d814005a-8155-4b28-b03f-8ce036e19757\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " Jul 28 17:12:22.355536 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.355361 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d814005a-8155-4b28-b03f-8ce036e19757-console-config\") pod \"d814005a-8155-4b28-b03f-8ce036e19757\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " Jul 28 17:12:22.355536 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.355383 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d814005a-8155-4b28-b03f-8ce036e19757-console-oauth-config\") pod \"d814005a-8155-4b28-b03f-8ce036e19757\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " Jul 28 17:12:22.355536 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.355459 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d814005a-8155-4b28-b03f-8ce036e19757-service-ca\") pod \"d814005a-8155-4b28-b03f-8ce036e19757\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " Jul 28 17:12:22.355536 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.355479 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d814005a-8155-4b28-b03f-8ce036e19757-oauth-serving-cert\") pod \"d814005a-8155-4b28-b03f-8ce036e19757\" (UID: \"d814005a-8155-4b28-b03f-8ce036e19757\") " Jul 28 17:12:22.355842 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.355807 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d814005a-8155-4b28-b03f-8ce036e19757-console-config" (OuterVolumeSpecName: "console-config") pod "d814005a-8155-4b28-b03f-8ce036e19757" (UID: "d814005a-8155-4b28-b03f-8ce036e19757"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:12:22.355960 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.355880 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d814005a-8155-4b28-b03f-8ce036e19757-service-ca" (OuterVolumeSpecName: "service-ca") pod "d814005a-8155-4b28-b03f-8ce036e19757" (UID: "d814005a-8155-4b28-b03f-8ce036e19757"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:12:22.356021 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.355994 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d814005a-8155-4b28-b03f-8ce036e19757-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d814005a-8155-4b28-b03f-8ce036e19757" (UID: "d814005a-8155-4b28-b03f-8ce036e19757"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:12:22.357659 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.357628 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d814005a-8155-4b28-b03f-8ce036e19757-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d814005a-8155-4b28-b03f-8ce036e19757" (UID: "d814005a-8155-4b28-b03f-8ce036e19757"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:12:22.357771 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.357748 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d814005a-8155-4b28-b03f-8ce036e19757-kube-api-access-pnmds" (OuterVolumeSpecName: "kube-api-access-pnmds") pod "d814005a-8155-4b28-b03f-8ce036e19757" (UID: "d814005a-8155-4b28-b03f-8ce036e19757"). InnerVolumeSpecName "kube-api-access-pnmds". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:12:22.357830 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.357751 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d814005a-8155-4b28-b03f-8ce036e19757-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d814005a-8155-4b28-b03f-8ce036e19757" (UID: "d814005a-8155-4b28-b03f-8ce036e19757"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:12:22.445319 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.445285 2550 generic.go:356] "Generic (PLEG): container finished" podID="d814005a-8155-4b28-b03f-8ce036e19757" containerID="91faed0def14316756885dfc08db33d0eeb896da99e04ee27f3187c9fbf22ca2" exitCode=0 Jul 28 17:12:22.445711 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.445331 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78d99d45cc-czwxg" Jul 28 17:12:22.445711 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.445370 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78d99d45cc-czwxg" event={"ID":"d814005a-8155-4b28-b03f-8ce036e19757","Type":"ContainerDied","Data":"91faed0def14316756885dfc08db33d0eeb896da99e04ee27f3187c9fbf22ca2"} Jul 28 17:12:22.445711 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.445448 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78d99d45cc-czwxg" event={"ID":"d814005a-8155-4b28-b03f-8ce036e19757","Type":"ContainerDied","Data":"a64f858e37970c32beb760e962078857eabc3107889e4c919ffd7e9230fb32a7"} Jul 28 17:12:22.445711 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.445475 2550 scope.go:117] "RemoveContainer" containerID="91faed0def14316756885dfc08db33d0eeb896da99e04ee27f3187c9fbf22ca2" Jul 28 17:12:22.454135 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.454112 2550 scope.go:117] "RemoveContainer" containerID="91faed0def14316756885dfc08db33d0eeb896da99e04ee27f3187c9fbf22ca2" Jul 28 17:12:22.454444 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:12:22.454424 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91faed0def14316756885dfc08db33d0eeb896da99e04ee27f3187c9fbf22ca2\": container with ID starting with 91faed0def14316756885dfc08db33d0eeb896da99e04ee27f3187c9fbf22ca2 not found: ID does not exist" containerID="91faed0def14316756885dfc08db33d0eeb896da99e04ee27f3187c9fbf22ca2" Jul 28 17:12:22.454507 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.454454 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91faed0def14316756885dfc08db33d0eeb896da99e04ee27f3187c9fbf22ca2"} err="failed to get container status \"91faed0def14316756885dfc08db33d0eeb896da99e04ee27f3187c9fbf22ca2\": rpc error: code = NotFound desc = could not find container \"91faed0def14316756885dfc08db33d0eeb896da99e04ee27f3187c9fbf22ca2\": container with ID starting with 91faed0def14316756885dfc08db33d0eeb896da99e04ee27f3187c9fbf22ca2 not found: ID does not exist" Jul 28 17:12:22.456820 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.456802 2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d814005a-8155-4b28-b03f-8ce036e19757-console-serving-cert\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:12:22.456872 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.456822 2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d814005a-8155-4b28-b03f-8ce036e19757-console-config\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:12:22.456872 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.456831 2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d814005a-8155-4b28-b03f-8ce036e19757-console-oauth-config\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:12:22.456872 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.456841 2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d814005a-8155-4b28-b03f-8ce036e19757-service-ca\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:12:22.456872 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.456849 2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d814005a-8155-4b28-b03f-8ce036e19757-oauth-serving-cert\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:12:22.456872 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.456857 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnmds\" (UniqueName: \"kubernetes.io/projected/d814005a-8155-4b28-b03f-8ce036e19757-kube-api-access-pnmds\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:12:22.468306 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.468279 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78d99d45cc-czwxg"] Jul 28 17:12:22.470600 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:22.470578 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78d99d45cc-czwxg"] Jul 28 17:12:23.771287 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:23.771253 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d814005a-8155-4b28-b03f-8ce036e19757" path="/var/lib/kubelet/pods/d814005a-8155-4b28-b03f-8ce036e19757/volumes" Jul 28 17:12:33.485427 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:33.485372 2550 generic.go:356] "Generic (PLEG): container finished" podID="73e0baae-ba8a-4ef3-aff6-a31782400af6" containerID="0b6575162f168c20c90e2e799f6cd9aaf2b638e7177ae93c07603a0ac39353f1" exitCode=0 Jul 28 17:12:33.485900 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:33.485446 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp" event={"ID":"73e0baae-ba8a-4ef3-aff6-a31782400af6","Type":"ContainerDied","Data":"0b6575162f168c20c90e2e799f6cd9aaf2b638e7177ae93c07603a0ac39353f1"} Jul 28 17:12:33.485900 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:33.485807 2550 scope.go:117] "RemoveContainer" containerID="0b6575162f168c20c90e2e799f6cd9aaf2b638e7177ae93c07603a0ac39353f1" Jul 28 17:12:33.487159 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:33.487049 2550 generic.go:356] "Generic (PLEG): container finished" podID="5a580271-8c91-43b9-8479-21fedddcf96f" containerID="65b6b9842b138caeedafe237052430c69d55ebab62162e33a6ff43c28b2c7443" exitCode=0 Jul 28 17:12:33.487159 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:33.487086 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-hzlqt" event={"ID":"5a580271-8c91-43b9-8479-21fedddcf96f","Type":"ContainerDied","Data":"65b6b9842b138caeedafe237052430c69d55ebab62162e33a6ff43c28b2c7443"} Jul 28 17:12:33.487364 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:33.487346 2550 scope.go:117] "RemoveContainer" containerID="65b6b9842b138caeedafe237052430c69d55ebab62162e33a6ff43c28b2c7443" Jul 28 17:12:34.492708 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:34.492673 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp" event={"ID":"73e0baae-ba8a-4ef3-aff6-a31782400af6","Type":"ContainerStarted","Data":"0530c4f3d00081ae54d50bc6259217f17649378cecfd54d2155c4d7ca90b47e2"} Jul 28 17:12:34.494486 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:12:34.494459 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-hzlqt" event={"ID":"5a580271-8c91-43b9-8479-21fedddcf96f","Type":"ContainerStarted","Data":"4585e4845fefb9c5a781b449e2b83348139694e659a5e3f53b55705df5d82e66"} Jul 28 17:13:07.520685 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.520648 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5647989fbd-lphn8"] Jul 28 17:13:07.521138 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.520922 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d814005a-8155-4b28-b03f-8ce036e19757" containerName="console" Jul 28 17:13:07.521138 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.520933 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="d814005a-8155-4b28-b03f-8ce036e19757" containerName="console" Jul 28 17:13:07.521138 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.520988 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="d814005a-8155-4b28-b03f-8ce036e19757" containerName="console" Jul 28 17:13:07.523692 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.523668 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.533542 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.533514 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5647989fbd-lphn8"] Jul 28 17:13:07.626556 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.626514 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85r8t\" (UniqueName: \"kubernetes.io/projected/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-kube-api-access-85r8t\") pod \"console-5647989fbd-lphn8\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.626556 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.626558 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-console-serving-cert\") pod \"console-5647989fbd-lphn8\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.626780 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.626584 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-console-oauth-config\") pod \"console-5647989fbd-lphn8\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.626780 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.626665 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-oauth-serving-cert\") pod \"console-5647989fbd-lphn8\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.626780 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.626698 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-console-config\") pod \"console-5647989fbd-lphn8\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.626780 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.626723 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-trusted-ca-bundle\") pod \"console-5647989fbd-lphn8\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.626932 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.626781 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-service-ca\") pod \"console-5647989fbd-lphn8\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.728129 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.728074 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-console-oauth-config\") pod \"console-5647989fbd-lphn8\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.728129 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.728145 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-oauth-serving-cert\") pod \"console-5647989fbd-lphn8\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.728431 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.728172 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-console-config\") pod \"console-5647989fbd-lphn8\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.728431 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.728201 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-trusted-ca-bundle\") pod \"console-5647989fbd-lphn8\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.728431 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.728246 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-service-ca\") pod \"console-5647989fbd-lphn8\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.728431 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.728310 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85r8t\" (UniqueName: \"kubernetes.io/projected/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-kube-api-access-85r8t\") pod \"console-5647989fbd-lphn8\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.728431 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.728335 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-console-serving-cert\") pod \"console-5647989fbd-lphn8\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.729072 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.729047 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-console-config\") pod \"console-5647989fbd-lphn8\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.729181 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.729124 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-oauth-serving-cert\") pod \"console-5647989fbd-lphn8\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.729244 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.729177 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-service-ca\") pod \"console-5647989fbd-lphn8\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.729298 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.729271 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-trusted-ca-bundle\") pod \"console-5647989fbd-lphn8\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.731169 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.731146 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-console-serving-cert\") pod \"console-5647989fbd-lphn8\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.731260 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.731167 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-console-oauth-config\") pod \"console-5647989fbd-lphn8\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.737325 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.737301 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85r8t\" (UniqueName: \"kubernetes.io/projected/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-kube-api-access-85r8t\") pod \"console-5647989fbd-lphn8\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.832968 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.832927 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:07.982874 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:07.982839 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5647989fbd-lphn8"] Jul 28 17:13:07.985960 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:13:07.985932 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod138fa9ba_85cc_4a9b_a7c8_3cbe29020d77.slice/crio-3fba17d9583df633296060eb6edd93917278c8e1dc42c488c67c772c8dd596e5 WatchSource:0}: Error finding container 3fba17d9583df633296060eb6edd93917278c8e1dc42c488c67c772c8dd596e5: Status 404 returned error can't find the container with id 3fba17d9583df633296060eb6edd93917278c8e1dc42c488c67c772c8dd596e5 Jul 28 17:13:08.622913 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:08.622874 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5647989fbd-lphn8" event={"ID":"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77","Type":"ContainerStarted","Data":"fef2cf5eeb5a2f37589ea2b1fb63bd310cd08ff863a34772a22c1e3b5eea6f93"} Jul 28 17:13:08.623304 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:08.622927 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5647989fbd-lphn8" event={"ID":"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77","Type":"ContainerStarted","Data":"3fba17d9583df633296060eb6edd93917278c8e1dc42c488c67c772c8dd596e5"} Jul 28 17:13:08.644029 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:08.643949 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5647989fbd-lphn8" podStartSLOduration=1.6439275100000001 podStartE2EDuration="1.64392751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:13:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:13:08.641599982 +0000 UTC m=+173.370101330" watchObservedRunningTime="2026-07-28 17:13:08.64392751 +0000 UTC m=+173.372428847" Jul 28 17:13:17.833918 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:17.833881 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:17.834431 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:17.833975 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:17.838697 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:17.838667 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:18.663284 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:18.663256 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:13:18.719242 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:18.719206 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6ccd4d6686-lkn2d"] Jul 28 17:13:43.738827 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:43.738777 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6ccd4d6686-lkn2d" podUID="1f81f663-9aac-471e-8ac2-6e05ceaf5111" containerName="console" containerID="cri-o://ea5b26108f7f89158bd6c043e44ffaf3a742efca6b30f46f18d808e238614be2" gracePeriod=15 Jul 28 17:13:43.979566 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:43.979536 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:13:44.004410 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.004321 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f81f663-9aac-471e-8ac2-6e05ceaf5111-console-oauth-config\") pod \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " Jul 28 17:13:44.004410 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.004363 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-oauth-serving-cert\") pod \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " Jul 28 17:13:44.004626 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.004469 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-service-ca\") pod \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " Jul 28 17:13:44.004626 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.004490 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-trusted-ca-bundle\") pod \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " Jul 28 17:13:44.004626 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.004532 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-console-config\") pod \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " Jul 28 17:13:44.004626 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.004555 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vkrm\" (UniqueName: \"kubernetes.io/projected/1f81f663-9aac-471e-8ac2-6e05ceaf5111-kube-api-access-8vkrm\") pod \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " Jul 28 17:13:44.004626 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.004585 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f81f663-9aac-471e-8ac2-6e05ceaf5111-console-serving-cert\") pod \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\" (UID: \"1f81f663-9aac-471e-8ac2-6e05ceaf5111\") " Jul 28 17:13:44.004904 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.004879 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1f81f663-9aac-471e-8ac2-6e05ceaf5111" (UID: "1f81f663-9aac-471e-8ac2-6e05ceaf5111"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:13:44.004971 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.004810 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-service-ca" (OuterVolumeSpecName: "service-ca") pod "1f81f663-9aac-471e-8ac2-6e05ceaf5111" (UID: "1f81f663-9aac-471e-8ac2-6e05ceaf5111"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:13:44.005013 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.004974 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-console-config" (OuterVolumeSpecName: "console-config") pod "1f81f663-9aac-471e-8ac2-6e05ceaf5111" (UID: "1f81f663-9aac-471e-8ac2-6e05ceaf5111"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:13:44.005013 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.004981 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1f81f663-9aac-471e-8ac2-6e05ceaf5111" (UID: "1f81f663-9aac-471e-8ac2-6e05ceaf5111"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:13:44.006763 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.006730 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f81f663-9aac-471e-8ac2-6e05ceaf5111-kube-api-access-8vkrm" (OuterVolumeSpecName: "kube-api-access-8vkrm") pod "1f81f663-9aac-471e-8ac2-6e05ceaf5111" (UID: "1f81f663-9aac-471e-8ac2-6e05ceaf5111"). InnerVolumeSpecName "kube-api-access-8vkrm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:13:44.006883 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.006803 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f81f663-9aac-471e-8ac2-6e05ceaf5111-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1f81f663-9aac-471e-8ac2-6e05ceaf5111" (UID: "1f81f663-9aac-471e-8ac2-6e05ceaf5111"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:13:44.006883 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.006830 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f81f663-9aac-471e-8ac2-6e05ceaf5111-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1f81f663-9aac-471e-8ac2-6e05ceaf5111" (UID: "1f81f663-9aac-471e-8ac2-6e05ceaf5111"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:13:44.105348 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.105311 2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f81f663-9aac-471e-8ac2-6e05ceaf5111-console-oauth-config\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:13:44.105348 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.105341 2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-oauth-serving-cert\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:13:44.105348 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.105351 2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-service-ca\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:13:44.105586 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.105359 2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-trusted-ca-bundle\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:13:44.105586 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.105368 2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f81f663-9aac-471e-8ac2-6e05ceaf5111-console-config\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:13:44.105586 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.105376 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vkrm\" (UniqueName: \"kubernetes.io/projected/1f81f663-9aac-471e-8ac2-6e05ceaf5111-kube-api-access-8vkrm\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:13:44.105586 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.105384 2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f81f663-9aac-471e-8ac2-6e05ceaf5111-console-serving-cert\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:13:44.753191 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.753153 2550 generic.go:356] "Generic (PLEG): container finished" podID="1f81f663-9aac-471e-8ac2-6e05ceaf5111" containerID="ea5b26108f7f89158bd6c043e44ffaf3a742efca6b30f46f18d808e238614be2" exitCode=0 Jul 28 17:13:44.753638 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.753203 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6ccd4d6686-lkn2d" Jul 28 17:13:44.753638 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.753245 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ccd4d6686-lkn2d" event={"ID":"1f81f663-9aac-471e-8ac2-6e05ceaf5111","Type":"ContainerDied","Data":"ea5b26108f7f89158bd6c043e44ffaf3a742efca6b30f46f18d808e238614be2"} Jul 28 17:13:44.753638 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.753297 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6ccd4d6686-lkn2d" event={"ID":"1f81f663-9aac-471e-8ac2-6e05ceaf5111","Type":"ContainerDied","Data":"9a3724d7096a2c1adbdbbc332db6e79c84a6b9ecd7b4bfa2a3a818d954e6f307"} Jul 28 17:13:44.753638 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.753322 2550 scope.go:117] "RemoveContainer" containerID="ea5b26108f7f89158bd6c043e44ffaf3a742efca6b30f46f18d808e238614be2" Jul 28 17:13:44.761754 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.761735 2550 scope.go:117] "RemoveContainer" containerID="ea5b26108f7f89158bd6c043e44ffaf3a742efca6b30f46f18d808e238614be2" Jul 28 17:13:44.762046 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:13:44.762025 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea5b26108f7f89158bd6c043e44ffaf3a742efca6b30f46f18d808e238614be2\": container with ID starting with ea5b26108f7f89158bd6c043e44ffaf3a742efca6b30f46f18d808e238614be2 not found: ID does not exist" containerID="ea5b26108f7f89158bd6c043e44ffaf3a742efca6b30f46f18d808e238614be2" Jul 28 17:13:44.762113 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.762052 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea5b26108f7f89158bd6c043e44ffaf3a742efca6b30f46f18d808e238614be2"} err="failed to get container status \"ea5b26108f7f89158bd6c043e44ffaf3a742efca6b30f46f18d808e238614be2\": rpc error: code = NotFound desc = could not find container \"ea5b26108f7f89158bd6c043e44ffaf3a742efca6b30f46f18d808e238614be2\": container with ID starting with ea5b26108f7f89158bd6c043e44ffaf3a742efca6b30f46f18d808e238614be2 not found: ID does not exist" Jul 28 17:13:44.775520 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.775491 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6ccd4d6686-lkn2d"] Jul 28 17:13:44.779036 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:44.779016 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6ccd4d6686-lkn2d"] Jul 28 17:13:45.771367 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:13:45.771321 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f81f663-9aac-471e-8ac2-6e05ceaf5111" path="/var/lib/kubelet/pods/1f81f663-9aac-471e-8ac2-6e05ceaf5111/volumes" Jul 28 17:14:15.495957 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.495919 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7bc455dd8f-6ztn4"] Jul 28 17:14:15.496463 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.496191 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f81f663-9aac-471e-8ac2-6e05ceaf5111" containerName="console" Jul 28 17:14:15.496463 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.496202 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f81f663-9aac-471e-8ac2-6e05ceaf5111" containerName="console" Jul 28 17:14:15.496463 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.496246 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f81f663-9aac-471e-8ac2-6e05ceaf5111" containerName="console" Jul 28 17:14:15.499189 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.499171 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.513294 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.513263 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bc455dd8f-6ztn4"] Jul 28 17:14:15.661879 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.661842 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmwvq\" (UniqueName: \"kubernetes.io/projected/e61b2270-bb80-4a76-a848-047004f7b43a-kube-api-access-pmwvq\") pod \"console-7bc455dd8f-6ztn4\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.661879 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.661883 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e61b2270-bb80-4a76-a848-047004f7b43a-console-serving-cert\") pod \"console-7bc455dd8f-6ztn4\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.662088 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.661903 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e61b2270-bb80-4a76-a848-047004f7b43a-console-oauth-config\") pod \"console-7bc455dd8f-6ztn4\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.662088 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.661919 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-oauth-serving-cert\") pod \"console-7bc455dd8f-6ztn4\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.662088 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.662029 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-service-ca\") pod \"console-7bc455dd8f-6ztn4\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.662088 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.662074 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-trusted-ca-bundle\") pod \"console-7bc455dd8f-6ztn4\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.662256 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.662115 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-console-config\") pod \"console-7bc455dd8f-6ztn4\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.762877 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.762780 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-console-config\") pod \"console-7bc455dd8f-6ztn4\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.762877 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.762829 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmwvq\" (UniqueName: \"kubernetes.io/projected/e61b2270-bb80-4a76-a848-047004f7b43a-kube-api-access-pmwvq\") pod \"console-7bc455dd8f-6ztn4\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.763125 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.762962 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e61b2270-bb80-4a76-a848-047004f7b43a-console-serving-cert\") pod \"console-7bc455dd8f-6ztn4\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.763125 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.763009 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e61b2270-bb80-4a76-a848-047004f7b43a-console-oauth-config\") pod \"console-7bc455dd8f-6ztn4\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.763125 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.763033 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-oauth-serving-cert\") pod \"console-7bc455dd8f-6ztn4\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.763272 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.763220 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-service-ca\") pod \"console-7bc455dd8f-6ztn4\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.763334 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.763276 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-trusted-ca-bundle\") pod \"console-7bc455dd8f-6ztn4\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.763634 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.763607 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-console-config\") pod \"console-7bc455dd8f-6ztn4\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.763845 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.763821 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-oauth-serving-cert\") pod \"console-7bc455dd8f-6ztn4\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.763917 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.763834 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-service-ca\") pod \"console-7bc455dd8f-6ztn4\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.764235 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.764215 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-trusted-ca-bundle\") pod \"console-7bc455dd8f-6ztn4\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.765498 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.765472 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e61b2270-bb80-4a76-a848-047004f7b43a-console-oauth-config\") pod \"console-7bc455dd8f-6ztn4\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.765584 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.765557 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e61b2270-bb80-4a76-a848-047004f7b43a-console-serving-cert\") pod \"console-7bc455dd8f-6ztn4\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.771496 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.771471 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmwvq\" (UniqueName: \"kubernetes.io/projected/e61b2270-bb80-4a76-a848-047004f7b43a-kube-api-access-pmwvq\") pod \"console-7bc455dd8f-6ztn4\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.808727 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.808689 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:15.943256 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:15.943232 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7bc455dd8f-6ztn4"] Jul 28 17:14:15.945684 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:14:15.945654 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode61b2270_bb80_4a76_a848_047004f7b43a.slice/crio-a8da411f54063e10b2ea87fa77f364f178a714f9bc7ca27f40508037c828d3b4 WatchSource:0}: Error finding container a8da411f54063e10b2ea87fa77f364f178a714f9bc7ca27f40508037c828d3b4: Status 404 returned error can't find the container with id a8da411f54063e10b2ea87fa77f364f178a714f9bc7ca27f40508037c828d3b4 Jul 28 17:14:16.867009 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:16.866966 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bc455dd8f-6ztn4" event={"ID":"e61b2270-bb80-4a76-a848-047004f7b43a","Type":"ContainerStarted","Data":"483c376a943b4767d1468c1f831b697d3a6bbea93809514a4ff0fb21d6c25135"} Jul 28 17:14:16.867424 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:16.867027 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bc455dd8f-6ztn4" event={"ID":"e61b2270-bb80-4a76-a848-047004f7b43a","Type":"ContainerStarted","Data":"a8da411f54063e10b2ea87fa77f364f178a714f9bc7ca27f40508037c828d3b4"} Jul 28 17:14:16.886005 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:16.885954 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7bc455dd8f-6ztn4" podStartSLOduration=1.885938315 podStartE2EDuration="1.885938315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:14:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:14:16.885260146 +0000 UTC m=+241.613761482" watchObservedRunningTime="2026-07-28 17:14:16.885938315 +0000 UTC m=+241.614439652" Jul 28 17:14:25.809499 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:25.809448 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:25.809910 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:25.809526 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:25.814176 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:25.814154 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:25.902884 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:25.902856 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:14:25.960407 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:25.960367 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5647989fbd-lphn8"] Jul 28 17:14:50.985292 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:50.985189 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5647989fbd-lphn8" podUID="138fa9ba-85cc-4a9b-a7c8-3cbe29020d77" containerName="console" containerID="cri-o://fef2cf5eeb5a2f37589ea2b1fb63bd310cd08ff863a34772a22c1e3b5eea6f93" gracePeriod=15 Jul 28 17:14:51.224332 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.224301 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:14:51.351285 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.351249 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-console-config\") pod \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " Jul 28 17:14:51.351285 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.351291 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-trusted-ca-bundle\") pod \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " Jul 28 17:14:51.351576 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.351311 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-oauth-serving-cert\") pod \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " Jul 28 17:14:51.351576 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.351335 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-service-ca\") pod \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " Jul 28 17:14:51.351576 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.351366 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-console-oauth-config\") pod \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " Jul 28 17:14:51.351576 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.351449 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-85r8t\" (UniqueName: \"kubernetes.io/projected/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-kube-api-access-85r8t\") pod \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " Jul 28 17:14:51.351576 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.351476 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-console-serving-cert\") pod \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\" (UID: \"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77\") " Jul 28 17:14:51.351827 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.351793 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-console-config" (OuterVolumeSpecName: "console-config") pod "138fa9ba-85cc-4a9b-a7c8-3cbe29020d77" (UID: "138fa9ba-85cc-4a9b-a7c8-3cbe29020d77"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:14:51.351894 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.351828 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "138fa9ba-85cc-4a9b-a7c8-3cbe29020d77" (UID: "138fa9ba-85cc-4a9b-a7c8-3cbe29020d77"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:14:51.351894 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.351837 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "138fa9ba-85cc-4a9b-a7c8-3cbe29020d77" (UID: "138fa9ba-85cc-4a9b-a7c8-3cbe29020d77"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:14:51.351894 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.351845 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-service-ca" (OuterVolumeSpecName: "service-ca") pod "138fa9ba-85cc-4a9b-a7c8-3cbe29020d77" (UID: "138fa9ba-85cc-4a9b-a7c8-3cbe29020d77"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:14:51.353643 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.353611 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-kube-api-access-85r8t" (OuterVolumeSpecName: "kube-api-access-85r8t") pod "138fa9ba-85cc-4a9b-a7c8-3cbe29020d77" (UID: "138fa9ba-85cc-4a9b-a7c8-3cbe29020d77"). InnerVolumeSpecName "kube-api-access-85r8t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:14:51.353747 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.353679 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "138fa9ba-85cc-4a9b-a7c8-3cbe29020d77" (UID: "138fa9ba-85cc-4a9b-a7c8-3cbe29020d77"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:14:51.353807 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.353791 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "138fa9ba-85cc-4a9b-a7c8-3cbe29020d77" (UID: "138fa9ba-85cc-4a9b-a7c8-3cbe29020d77"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:14:51.452962 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.452912 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-85r8t\" (UniqueName: \"kubernetes.io/projected/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-kube-api-access-85r8t\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:14:51.452962 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.452955 2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-console-serving-cert\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:14:51.452962 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.452967 2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-console-config\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:14:51.452962 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.452976 2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-trusted-ca-bundle\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:14:51.453212 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.452984 2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-oauth-serving-cert\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:14:51.453212 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.452992 2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-service-ca\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:14:51.453212 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.453003 2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77-console-oauth-config\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:14:51.997749 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.997716 2550 generic.go:356] "Generic (PLEG): container finished" podID="138fa9ba-85cc-4a9b-a7c8-3cbe29020d77" containerID="fef2cf5eeb5a2f37589ea2b1fb63bd310cd08ff863a34772a22c1e3b5eea6f93" exitCode=0 Jul 28 17:14:51.998161 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.997763 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5647989fbd-lphn8" Jul 28 17:14:51.998161 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.997775 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5647989fbd-lphn8" event={"ID":"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77","Type":"ContainerDied","Data":"fef2cf5eeb5a2f37589ea2b1fb63bd310cd08ff863a34772a22c1e3b5eea6f93"} Jul 28 17:14:51.998161 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.997815 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5647989fbd-lphn8" event={"ID":"138fa9ba-85cc-4a9b-a7c8-3cbe29020d77","Type":"ContainerDied","Data":"3fba17d9583df633296060eb6edd93917278c8e1dc42c488c67c772c8dd596e5"} Jul 28 17:14:51.998161 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:51.998123 2550 scope.go:117] "RemoveContainer" containerID="fef2cf5eeb5a2f37589ea2b1fb63bd310cd08ff863a34772a22c1e3b5eea6f93" Jul 28 17:14:52.011181 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:52.011163 2550 scope.go:117] "RemoveContainer" containerID="fef2cf5eeb5a2f37589ea2b1fb63bd310cd08ff863a34772a22c1e3b5eea6f93" Jul 28 17:14:52.011464 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:14:52.011444 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fef2cf5eeb5a2f37589ea2b1fb63bd310cd08ff863a34772a22c1e3b5eea6f93\": container with ID starting with fef2cf5eeb5a2f37589ea2b1fb63bd310cd08ff863a34772a22c1e3b5eea6f93 not found: ID does not exist" containerID="fef2cf5eeb5a2f37589ea2b1fb63bd310cd08ff863a34772a22c1e3b5eea6f93" Jul 28 17:14:52.011517 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:52.011472 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fef2cf5eeb5a2f37589ea2b1fb63bd310cd08ff863a34772a22c1e3b5eea6f93"} err="failed to get container status \"fef2cf5eeb5a2f37589ea2b1fb63bd310cd08ff863a34772a22c1e3b5eea6f93\": rpc error: code = NotFound desc = could not find container \"fef2cf5eeb5a2f37589ea2b1fb63bd310cd08ff863a34772a22c1e3b5eea6f93\": container with ID starting with fef2cf5eeb5a2f37589ea2b1fb63bd310cd08ff863a34772a22c1e3b5eea6f93 not found: ID does not exist" Jul 28 17:14:52.018091 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:52.018065 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5647989fbd-lphn8"] Jul 28 17:14:52.019772 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:52.019754 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5647989fbd-lphn8"] Jul 28 17:14:53.771609 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:14:53.771571 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="138fa9ba-85cc-4a9b-a7c8-3cbe29020d77" path="/var/lib/kubelet/pods/138fa9ba-85cc-4a9b-a7c8-3cbe29020d77/volumes" Jul 28 17:15:15.611066 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:15.611034 2550 kubelet.go:1648] "Image garbage collection succeeded" Jul 28 17:15:31.747810 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:31.747776 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4"] Jul 28 17:15:31.750195 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:31.748036 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="138fa9ba-85cc-4a9b-a7c8-3cbe29020d77" containerName="console" Jul 28 17:15:31.750195 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:31.748057 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="138fa9ba-85cc-4a9b-a7c8-3cbe29020d77" containerName="console" Jul 28 17:15:31.750195 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:31.748103 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="138fa9ba-85cc-4a9b-a7c8-3cbe29020d77" containerName="console" Jul 28 17:15:31.751049 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:31.751033 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4" Jul 28 17:15:31.754072 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:31.754050 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-psmt4\"" Jul 28 17:15:31.754201 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:31.754076 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 28 17:15:31.754201 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:31.754088 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 28 17:15:31.760746 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:31.760722 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4"] Jul 28 17:15:31.874333 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:31.874274 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a2c0b8d4-f354-4958-a082-767b840eb9cd-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4\" (UID: \"a2c0b8d4-f354-4958-a082-767b840eb9cd\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4" Jul 28 17:15:31.874576 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:31.874373 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dprmq\" (UniqueName: \"kubernetes.io/projected/a2c0b8d4-f354-4958-a082-767b840eb9cd-kube-api-access-dprmq\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4\" (UID: \"a2c0b8d4-f354-4958-a082-767b840eb9cd\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4" Jul 28 17:15:31.874576 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:31.874519 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a2c0b8d4-f354-4958-a082-767b840eb9cd-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4\" (UID: \"a2c0b8d4-f354-4958-a082-767b840eb9cd\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4" Jul 28 17:15:31.975552 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:31.975503 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a2c0b8d4-f354-4958-a082-767b840eb9cd-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4\" (UID: \"a2c0b8d4-f354-4958-a082-767b840eb9cd\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4" Jul 28 17:15:31.975552 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:31.975559 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dprmq\" (UniqueName: \"kubernetes.io/projected/a2c0b8d4-f354-4958-a082-767b840eb9cd-kube-api-access-dprmq\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4\" (UID: \"a2c0b8d4-f354-4958-a082-767b840eb9cd\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4" Jul 28 17:15:31.975756 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:31.975597 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a2c0b8d4-f354-4958-a082-767b840eb9cd-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4\" (UID: \"a2c0b8d4-f354-4958-a082-767b840eb9cd\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4" Jul 28 17:15:31.975899 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:31.975878 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a2c0b8d4-f354-4958-a082-767b840eb9cd-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4\" (UID: \"a2c0b8d4-f354-4958-a082-767b840eb9cd\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4" Jul 28 17:15:31.975933 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:31.975903 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a2c0b8d4-f354-4958-a082-767b840eb9cd-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4\" (UID: \"a2c0b8d4-f354-4958-a082-767b840eb9cd\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4" Jul 28 17:15:31.986481 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:31.986453 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dprmq\" (UniqueName: \"kubernetes.io/projected/a2c0b8d4-f354-4958-a082-767b840eb9cd-kube-api-access-dprmq\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4\" (UID: \"a2c0b8d4-f354-4958-a082-767b840eb9cd\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4" Jul 28 17:15:32.060512 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:32.060465 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4" Jul 28 17:15:32.191759 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:32.191722 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4"] Jul 28 17:15:32.195023 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:15:32.194984 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2c0b8d4_f354_4958_a082_767b840eb9cd.slice/crio-cf3b20fc1316daae0c6b54495bf6728d49c7af2fe52a44d836eea1e8e527a9c7 WatchSource:0}: Error finding container cf3b20fc1316daae0c6b54495bf6728d49c7af2fe52a44d836eea1e8e527a9c7: Status 404 returned error can't find the container with id cf3b20fc1316daae0c6b54495bf6728d49c7af2fe52a44d836eea1e8e527a9c7 Jul 28 17:15:32.196947 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:32.196930 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:15:33.146236 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:33.146178 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4" event={"ID":"a2c0b8d4-f354-4958-a082-767b840eb9cd","Type":"ContainerStarted","Data":"cf3b20fc1316daae0c6b54495bf6728d49c7af2fe52a44d836eea1e8e527a9c7"} Jul 28 17:15:38.166203 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:38.166162 2550 generic.go:356] "Generic (PLEG): container finished" podID="a2c0b8d4-f354-4958-a082-767b840eb9cd" containerID="26599df20244a8b1604247e2dde3bde828207e3270b1f979f38f8841514e2cb4" exitCode=0 Jul 28 17:15:38.166596 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:38.166231 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4" event={"ID":"a2c0b8d4-f354-4958-a082-767b840eb9cd","Type":"ContainerDied","Data":"26599df20244a8b1604247e2dde3bde828207e3270b1f979f38f8841514e2cb4"} Jul 28 17:15:40.175523 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:40.175494 2550 generic.go:356] "Generic (PLEG): container finished" podID="a2c0b8d4-f354-4958-a082-767b840eb9cd" containerID="c5b6e5b0040ad8684fd676450f956fa2c54cc9b52c33b27bac7fc542948ee452" exitCode=0 Jul 28 17:15:40.175903 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:40.175558 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4" event={"ID":"a2c0b8d4-f354-4958-a082-767b840eb9cd","Type":"ContainerDied","Data":"c5b6e5b0040ad8684fd676450f956fa2c54cc9b52c33b27bac7fc542948ee452"} Jul 28 17:15:47.204663 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:47.204572 2550 generic.go:356] "Generic (PLEG): container finished" podID="a2c0b8d4-f354-4958-a082-767b840eb9cd" containerID="8967632a7f27405c72b1f6bfd1fe21303b842c2db7164470bdb1718d6dac83f6" exitCode=0 Jul 28 17:15:47.204663 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:47.204617 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4" event={"ID":"a2c0b8d4-f354-4958-a082-767b840eb9cd","Type":"ContainerDied","Data":"8967632a7f27405c72b1f6bfd1fe21303b842c2db7164470bdb1718d6dac83f6"} Jul 28 17:15:48.332184 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:48.332160 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4" Jul 28 17:15:48.411481 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:48.411449 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a2c0b8d4-f354-4958-a082-767b840eb9cd-bundle\") pod \"a2c0b8d4-f354-4958-a082-767b840eb9cd\" (UID: \"a2c0b8d4-f354-4958-a082-767b840eb9cd\") " Jul 28 17:15:48.411649 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:48.411504 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dprmq\" (UniqueName: \"kubernetes.io/projected/a2c0b8d4-f354-4958-a082-767b840eb9cd-kube-api-access-dprmq\") pod \"a2c0b8d4-f354-4958-a082-767b840eb9cd\" (UID: \"a2c0b8d4-f354-4958-a082-767b840eb9cd\") " Jul 28 17:15:48.411649 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:48.411530 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a2c0b8d4-f354-4958-a082-767b840eb9cd-util\") pod \"a2c0b8d4-f354-4958-a082-767b840eb9cd\" (UID: \"a2c0b8d4-f354-4958-a082-767b840eb9cd\") " Jul 28 17:15:48.412125 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:48.412096 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2c0b8d4-f354-4958-a082-767b840eb9cd-bundle" (OuterVolumeSpecName: "bundle") pod "a2c0b8d4-f354-4958-a082-767b840eb9cd" (UID: "a2c0b8d4-f354-4958-a082-767b840eb9cd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:15:48.413744 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:48.413722 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2c0b8d4-f354-4958-a082-767b840eb9cd-kube-api-access-dprmq" (OuterVolumeSpecName: "kube-api-access-dprmq") pod "a2c0b8d4-f354-4958-a082-767b840eb9cd" (UID: "a2c0b8d4-f354-4958-a082-767b840eb9cd"). InnerVolumeSpecName "kube-api-access-dprmq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:15:48.416174 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:48.416150 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2c0b8d4-f354-4958-a082-767b840eb9cd-util" (OuterVolumeSpecName: "util") pod "a2c0b8d4-f354-4958-a082-767b840eb9cd" (UID: "a2c0b8d4-f354-4958-a082-767b840eb9cd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 17:15:48.512827 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:48.512728 2550 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a2c0b8d4-f354-4958-a082-767b840eb9cd-bundle\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:15:48.512827 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:48.512771 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dprmq\" (UniqueName: \"kubernetes.io/projected/a2c0b8d4-f354-4958-a082-767b840eb9cd-kube-api-access-dprmq\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:15:48.512827 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:48.512781 2550 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a2c0b8d4-f354-4958-a082-767b840eb9cd-util\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:15:49.213915 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:49.213889 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4" Jul 28 17:15:49.214084 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:49.213885 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5bj2j4" event={"ID":"a2c0b8d4-f354-4958-a082-767b840eb9cd","Type":"ContainerDied","Data":"cf3b20fc1316daae0c6b54495bf6728d49c7af2fe52a44d836eea1e8e527a9c7"} Jul 28 17:15:49.214084 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:49.214006 2550 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cf3b20fc1316daae0c6b54495bf6728d49c7af2fe52a44d836eea1e8e527a9c7" Jul 28 17:15:56.006708 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.006674 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr"] Jul 28 17:15:56.007107 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.006942 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2c0b8d4-f354-4958-a082-767b840eb9cd" containerName="pull" Jul 28 17:15:56.007107 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.006953 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2c0b8d4-f354-4958-a082-767b840eb9cd" containerName="pull" Jul 28 17:15:56.007107 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.006972 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2c0b8d4-f354-4958-a082-767b840eb9cd" containerName="extract" Jul 28 17:15:56.007107 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.006978 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2c0b8d4-f354-4958-a082-767b840eb9cd" containerName="extract" Jul 28 17:15:56.007107 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.006989 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2c0b8d4-f354-4958-a082-767b840eb9cd" containerName="util" Jul 28 17:15:56.007107 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.006994 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2c0b8d4-f354-4958-a082-767b840eb9cd" containerName="util" Jul 28 17:15:56.007107 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.007036 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2c0b8d4-f354-4958-a082-767b840eb9cd" containerName="extract" Jul 28 17:15:56.051274 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.051235 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr"] Jul 28 17:15:56.051483 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.051376 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr" Jul 28 17:15:56.055031 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.054999 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Jul 28 17:15:56.055172 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.055006 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Jul 28 17:15:56.055172 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.055083 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Jul 28 17:15:56.055316 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.055296 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-xfxld\"" Jul 28 17:15:56.173952 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.173915 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/2bb37185-0eed-4d39-954a-20648edee7a6-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr\" (UID: \"2bb37185-0eed-4d39-954a-20648edee7a6\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr" Jul 28 17:15:56.173952 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.173955 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcn4k\" (UniqueName: \"kubernetes.io/projected/2bb37185-0eed-4d39-954a-20648edee7a6-kube-api-access-wcn4k\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr\" (UID: \"2bb37185-0eed-4d39-954a-20648edee7a6\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr" Jul 28 17:15:56.275240 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.275143 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/2bb37185-0eed-4d39-954a-20648edee7a6-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr\" (UID: \"2bb37185-0eed-4d39-954a-20648edee7a6\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr" Jul 28 17:15:56.275240 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.275189 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcn4k\" (UniqueName: \"kubernetes.io/projected/2bb37185-0eed-4d39-954a-20648edee7a6-kube-api-access-wcn4k\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr\" (UID: \"2bb37185-0eed-4d39-954a-20648edee7a6\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr" Jul 28 17:15:56.277508 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.277485 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/2bb37185-0eed-4d39-954a-20648edee7a6-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr\" (UID: \"2bb37185-0eed-4d39-954a-20648edee7a6\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr" Jul 28 17:15:56.285031 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.285009 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcn4k\" (UniqueName: \"kubernetes.io/projected/2bb37185-0eed-4d39-954a-20648edee7a6-kube-api-access-wcn4k\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr\" (UID: \"2bb37185-0eed-4d39-954a-20648edee7a6\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr" Jul 28 17:15:56.362041 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.361997 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr" Jul 28 17:15:56.515499 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:56.515471 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr"] Jul 28 17:15:56.516417 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:15:56.516375 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bb37185_0eed_4d39_954a_20648edee7a6.slice/crio-f8789a97d0feb489f54687ffb99504117f6fb6efc93a0a66809e030864169572 WatchSource:0}: Error finding container f8789a97d0feb489f54687ffb99504117f6fb6efc93a0a66809e030864169572: Status 404 returned error can't find the container with id f8789a97d0feb489f54687ffb99504117f6fb6efc93a0a66809e030864169572 Jul 28 17:15:57.246070 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:15:57.246035 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr" event={"ID":"2bb37185-0eed-4d39-954a-20648edee7a6","Type":"ContainerStarted","Data":"f8789a97d0feb489f54687ffb99504117f6fb6efc93a0a66809e030864169572"} Jul 28 17:16:00.258708 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.258670 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr" event={"ID":"2bb37185-0eed-4d39-954a-20648edee7a6","Type":"ContainerStarted","Data":"20f2272010beda946f1b5dade6cbbed049d8c03fdbcabbb853d6994ad34cae02"} Jul 28 17:16:00.259119 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.258797 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr" Jul 28 17:16:00.285494 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.285431 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr" podStartSLOduration=2.061394904 podStartE2EDuration="5.285415309s" totalImagesPullingTime="3.224020405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:15:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:16:00.28422086 +0000 UTC m=+345.012722219" watchObservedRunningTime="2026-07-28 17:16:00.285415309 +0000 UTC m=+345.013916645" Jul 28 17:16:00.617960 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.617925 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-77f648d694-zbszp"] Jul 28 17:16:00.639024 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.638984 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-zbszp"] Jul 28 17:16:00.639162 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.639145 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-zbszp" Jul 28 17:16:00.642337 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.642316 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\"" Jul 28 17:16:00.642479 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.642317 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Jul 28 17:16:00.642629 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.642616 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-rmfkh\"" Jul 28 17:16:00.716399 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.716353 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mww4t\" (UniqueName: \"kubernetes.io/projected/c326a324-a578-43f8-9e79-ef7a9c22bfa3-kube-api-access-mww4t\") pod \"keda-operator-77f648d694-zbszp\" (UID: \"c326a324-a578-43f8-9e79-ef7a9c22bfa3\") " pod="openshift-keda/keda-operator-77f648d694-zbszp" Jul 28 17:16:00.716563 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.716424 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/c326a324-a578-43f8-9e79-ef7a9c22bfa3-cabundle0\") pod \"keda-operator-77f648d694-zbszp\" (UID: \"c326a324-a578-43f8-9e79-ef7a9c22bfa3\") " pod="openshift-keda/keda-operator-77f648d694-zbszp" Jul 28 17:16:00.716563 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.716459 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c326a324-a578-43f8-9e79-ef7a9c22bfa3-certificates\") pod \"keda-operator-77f648d694-zbszp\" (UID: \"c326a324-a578-43f8-9e79-ef7a9c22bfa3\") " pod="openshift-keda/keda-operator-77f648d694-zbszp" Jul 28 17:16:00.817087 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.817049 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mww4t\" (UniqueName: \"kubernetes.io/projected/c326a324-a578-43f8-9e79-ef7a9c22bfa3-kube-api-access-mww4t\") pod \"keda-operator-77f648d694-zbszp\" (UID: \"c326a324-a578-43f8-9e79-ef7a9c22bfa3\") " pod="openshift-keda/keda-operator-77f648d694-zbszp" Jul 28 17:16:00.817298 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.817134 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/c326a324-a578-43f8-9e79-ef7a9c22bfa3-cabundle0\") pod \"keda-operator-77f648d694-zbszp\" (UID: \"c326a324-a578-43f8-9e79-ef7a9c22bfa3\") " pod="openshift-keda/keda-operator-77f648d694-zbszp" Jul 28 17:16:00.817298 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.817167 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c326a324-a578-43f8-9e79-ef7a9c22bfa3-certificates\") pod \"keda-operator-77f648d694-zbszp\" (UID: \"c326a324-a578-43f8-9e79-ef7a9c22bfa3\") " pod="openshift-keda/keda-operator-77f648d694-zbszp" Jul 28 17:16:00.817506 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:16:00.817488 2550 secret.go:281] references non-existent secret key: ca.crt Jul 28 17:16:00.817567 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:16:00.817512 2550 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Jul 28 17:16:00.817567 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:16:00.817524 2550 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-77f648d694-zbszp: references non-existent secret key: ca.crt Jul 28 17:16:00.817665 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:16:00.817587 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c326a324-a578-43f8-9e79-ef7a9c22bfa3-certificates podName:c326a324-a578-43f8-9e79-ef7a9c22bfa3 nodeName:}" failed. No retries permitted until 2026-07-28 17:16:01.317567343 +0000 UTC m=+346.046068671 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/c326a324-a578-43f8-9e79-ef7a9c22bfa3-certificates") pod "keda-operator-77f648d694-zbszp" (UID: "c326a324-a578-43f8-9e79-ef7a9c22bfa3") : references non-existent secret key: ca.crt Jul 28 17:16:00.817835 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.817815 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/c326a324-a578-43f8-9e79-ef7a9c22bfa3-cabundle0\") pod \"keda-operator-77f648d694-zbszp\" (UID: \"c326a324-a578-43f8-9e79-ef7a9c22bfa3\") " pod="openshift-keda/keda-operator-77f648d694-zbszp" Jul 28 17:16:00.831739 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.831705 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-9v6bf"] Jul 28 17:16:00.834418 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.834379 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mww4t\" (UniqueName: \"kubernetes.io/projected/c326a324-a578-43f8-9e79-ef7a9c22bfa3-kube-api-access-mww4t\") pod \"keda-operator-77f648d694-zbszp\" (UID: \"c326a324-a578-43f8-9e79-ef7a9c22bfa3\") " pod="openshift-keda/keda-operator-77f648d694-zbszp" Jul 28 17:16:00.853761 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.853719 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-9v6bf"] Jul 28 17:16:00.853906 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.853836 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-9v6bf" Jul 28 17:16:00.856741 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.856719 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" Jul 28 17:16:00.918078 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.917778 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863-certificates\") pod \"keda-metrics-apiserver-55557df8c-9v6bf\" (UID: \"c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-9v6bf" Jul 28 17:16:00.918078 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.917839 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-9v6bf\" (UID: \"c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-9v6bf" Jul 28 17:16:00.918078 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:00.917914 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfb6v\" (UniqueName: \"kubernetes.io/projected/c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863-kube-api-access-zfb6v\") pod \"keda-metrics-apiserver-55557df8c-9v6bf\" (UID: \"c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-9v6bf" Jul 28 17:16:01.021134 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.019782 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfb6v\" (UniqueName: \"kubernetes.io/projected/c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863-kube-api-access-zfb6v\") pod \"keda-metrics-apiserver-55557df8c-9v6bf\" (UID: \"c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-9v6bf" Jul 28 17:16:01.021134 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.019889 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863-certificates\") pod \"keda-metrics-apiserver-55557df8c-9v6bf\" (UID: \"c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-9v6bf" Jul 28 17:16:01.021134 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.019937 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-9v6bf\" (UID: \"c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-9v6bf" Jul 28 17:16:01.021134 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.020421 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-9v6bf\" (UID: \"c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-9v6bf" Jul 28 17:16:01.021134 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:16:01.020868 2550 secret.go:281] references non-existent secret key: tls.crt Jul 28 17:16:01.021134 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:16:01.020887 2550 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt Jul 28 17:16:01.021134 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:16:01.020907 2550 projected.go:266] Couldn't get secret openshift-keda/keda-metrics-apiserver-certs: secret "keda-metrics-apiserver-certs" not found Jul 28 17:16:01.021134 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:16:01.020938 2550 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-55557df8c-9v6bf: [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found] Jul 28 17:16:01.021134 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:16:01.021005 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863-certificates podName:c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863 nodeName:}" failed. No retries permitted until 2026-07-28 17:16:01.520978838 +0000 UTC m=+346.249480153 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863-certificates") pod "keda-metrics-apiserver-55557df8c-9v6bf" (UID: "c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863") : [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found] Jul 28 17:16:01.029804 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.029779 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfb6v\" (UniqueName: \"kubernetes.io/projected/c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863-kube-api-access-zfb6v\") pod \"keda-metrics-apiserver-55557df8c-9v6bf\" (UID: \"c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-9v6bf" Jul 28 17:16:01.146148 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.146111 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-9mvl5"] Jul 28 17:16:01.148368 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.148349 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-9mvl5" Jul 28 17:16:01.151176 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.151155 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" Jul 28 17:16:01.161760 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.161728 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-9mvl5"] Jul 28 17:16:01.222269 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.222178 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/cd488131-b959-4f8a-bf43-375ba0719f7b-certificates\") pod \"keda-admission-56f9798bb4-9mvl5\" (UID: \"cd488131-b959-4f8a-bf43-375ba0719f7b\") " pod="openshift-keda/keda-admission-56f9798bb4-9mvl5" Jul 28 17:16:01.222269 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.222241 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tk9vj\" (UniqueName: \"kubernetes.io/projected/cd488131-b959-4f8a-bf43-375ba0719f7b-kube-api-access-tk9vj\") pod \"keda-admission-56f9798bb4-9mvl5\" (UID: \"cd488131-b959-4f8a-bf43-375ba0719f7b\") " pod="openshift-keda/keda-admission-56f9798bb4-9mvl5" Jul 28 17:16:01.322685 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.322652 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c326a324-a578-43f8-9e79-ef7a9c22bfa3-certificates\") pod \"keda-operator-77f648d694-zbszp\" (UID: \"c326a324-a578-43f8-9e79-ef7a9c22bfa3\") " pod="openshift-keda/keda-operator-77f648d694-zbszp" Jul 28 17:16:01.323186 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.322715 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/cd488131-b959-4f8a-bf43-375ba0719f7b-certificates\") pod \"keda-admission-56f9798bb4-9mvl5\" (UID: \"cd488131-b959-4f8a-bf43-375ba0719f7b\") " pod="openshift-keda/keda-admission-56f9798bb4-9mvl5" Jul 28 17:16:01.323186 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.322782 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tk9vj\" (UniqueName: \"kubernetes.io/projected/cd488131-b959-4f8a-bf43-375ba0719f7b-kube-api-access-tk9vj\") pod \"keda-admission-56f9798bb4-9mvl5\" (UID: \"cd488131-b959-4f8a-bf43-375ba0719f7b\") " pod="openshift-keda/keda-admission-56f9798bb4-9mvl5" Jul 28 17:16:01.323186 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:16:01.322812 2550 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found Jul 28 17:16:01.323186 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:16:01.322839 2550 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-56f9798bb4-9mvl5: secret "keda-admission-webhooks-certs" not found Jul 28 17:16:01.323186 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:16:01.322893 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd488131-b959-4f8a-bf43-375ba0719f7b-certificates podName:cd488131-b959-4f8a-bf43-375ba0719f7b nodeName:}" failed. No retries permitted until 2026-07-28 17:16:01.822872722 +0000 UTC m=+346.551374067 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/cd488131-b959-4f8a-bf43-375ba0719f7b-certificates") pod "keda-admission-56f9798bb4-9mvl5" (UID: "cd488131-b959-4f8a-bf43-375ba0719f7b") : secret "keda-admission-webhooks-certs" not found Jul 28 17:16:01.325126 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.325101 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c326a324-a578-43f8-9e79-ef7a9c22bfa3-certificates\") pod \"keda-operator-77f648d694-zbszp\" (UID: \"c326a324-a578-43f8-9e79-ef7a9c22bfa3\") " pod="openshift-keda/keda-operator-77f648d694-zbszp" Jul 28 17:16:01.333007 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.332987 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tk9vj\" (UniqueName: \"kubernetes.io/projected/cd488131-b959-4f8a-bf43-375ba0719f7b-kube-api-access-tk9vj\") pod \"keda-admission-56f9798bb4-9mvl5\" (UID: \"cd488131-b959-4f8a-bf43-375ba0719f7b\") " pod="openshift-keda/keda-admission-56f9798bb4-9mvl5" Jul 28 17:16:01.524823 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.524725 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863-certificates\") pod \"keda-metrics-apiserver-55557df8c-9v6bf\" (UID: \"c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-9v6bf" Jul 28 17:16:01.527653 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.527627 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863-certificates\") pod \"keda-metrics-apiserver-55557df8c-9v6bf\" (UID: \"c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-9v6bf" Jul 28 17:16:01.548589 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.548554 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-zbszp" Jul 28 17:16:01.680208 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.680175 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-zbszp"] Jul 28 17:16:01.683133 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:16:01.683107 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc326a324_a578_43f8_9e79_ef7a9c22bfa3.slice/crio-256af7c13bd647bd4a93f463de91359a5837219271215dbd682c14311a10fae6 WatchSource:0}: Error finding container 256af7c13bd647bd4a93f463de91359a5837219271215dbd682c14311a10fae6: Status 404 returned error can't find the container with id 256af7c13bd647bd4a93f463de91359a5837219271215dbd682c14311a10fae6 Jul 28 17:16:01.764750 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.764713 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-9v6bf" Jul 28 17:16:01.827355 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.827324 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/cd488131-b959-4f8a-bf43-375ba0719f7b-certificates\") pod \"keda-admission-56f9798bb4-9mvl5\" (UID: \"cd488131-b959-4f8a-bf43-375ba0719f7b\") " pod="openshift-keda/keda-admission-56f9798bb4-9mvl5" Jul 28 17:16:01.829828 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.829796 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/cd488131-b959-4f8a-bf43-375ba0719f7b-certificates\") pod \"keda-admission-56f9798bb4-9mvl5\" (UID: \"cd488131-b959-4f8a-bf43-375ba0719f7b\") " pod="openshift-keda/keda-admission-56f9798bb4-9mvl5" Jul 28 17:16:01.899946 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:01.899911 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-9v6bf"] Jul 28 17:16:01.902685 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:16:01.902653 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6ed3dd2_a13a_4f7e_8631_a88e4b6c5863.slice/crio-9b35bc32d9e722c30dcd59952bdf30d2c5925ff8c376a26a542ab97ab8a0b117 WatchSource:0}: Error finding container 9b35bc32d9e722c30dcd59952bdf30d2c5925ff8c376a26a542ab97ab8a0b117: Status 404 returned error can't find the container with id 9b35bc32d9e722c30dcd59952bdf30d2c5925ff8c376a26a542ab97ab8a0b117 Jul 28 17:16:02.058523 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:02.058483 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-9mvl5" Jul 28 17:16:02.192171 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:02.192035 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-9mvl5"] Jul 28 17:16:02.195511 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:16:02.195467 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd488131_b959_4f8a_bf43_375ba0719f7b.slice/crio-5a77b8bd37ce16cf89c85f348aa72f59ab36e5d5bdd73677bbc068bce3b1f3c5 WatchSource:0}: Error finding container 5a77b8bd37ce16cf89c85f348aa72f59ab36e5d5bdd73677bbc068bce3b1f3c5: Status 404 returned error can't find the container with id 5a77b8bd37ce16cf89c85f348aa72f59ab36e5d5bdd73677bbc068bce3b1f3c5 Jul 28 17:16:02.269849 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:02.269806 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-9v6bf" event={"ID":"c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863","Type":"ContainerStarted","Data":"9b35bc32d9e722c30dcd59952bdf30d2c5925ff8c376a26a542ab97ab8a0b117"} Jul 28 17:16:02.271209 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:02.271158 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-9mvl5" event={"ID":"cd488131-b959-4f8a-bf43-375ba0719f7b","Type":"ContainerStarted","Data":"5a77b8bd37ce16cf89c85f348aa72f59ab36e5d5bdd73677bbc068bce3b1f3c5"} Jul 28 17:16:02.272366 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:02.272342 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-zbszp" event={"ID":"c326a324-a578-43f8-9e79-ef7a9c22bfa3","Type":"ContainerStarted","Data":"256af7c13bd647bd4a93f463de91359a5837219271215dbd682c14311a10fae6"} Jul 28 17:16:04.286623 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:04.286578 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-9mvl5" event={"ID":"cd488131-b959-4f8a-bf43-375ba0719f7b","Type":"ContainerStarted","Data":"bdffd6cb63e85bc25ba51042d1f716dcd445da66fc7ca7ffb29406cd0518fd80"} Jul 28 17:16:04.287067 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:04.286684 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-56f9798bb4-9mvl5" Jul 28 17:16:04.315909 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:04.314730 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-56f9798bb4-9mvl5" podStartSLOduration=1.9530661679999999 podStartE2EDuration="3.314709275s" totalImagesPullingTime="1.361643107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:16:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:16:04.314199157 +0000 UTC m=+349.042700494" watchObservedRunningTime="2026-07-28 17:16:04.314709275 +0000 UTC m=+349.043210614" Jul 28 17:16:06.299142 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:06.299099 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-zbszp" event={"ID":"c326a324-a578-43f8-9e79-ef7a9c22bfa3","Type":"ContainerStarted","Data":"629f845851c8405b046874d81ba94946c33210f0543f6b6ab9e26e02754d48f5"} Jul 28 17:16:06.299617 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:06.299208 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-77f648d694-zbszp" Jul 28 17:16:06.300640 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:06.300617 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-9v6bf" event={"ID":"c6ed3dd2-a13a-4f7e-8631-a88e4b6c5863","Type":"ContainerStarted","Data":"d3ada8a121bbfae830a8cd3d8b3ca9f5c47a17de97e26dda545ca717aa414e2a"} Jul 28 17:16:06.300751 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:06.300735 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-9v6bf" Jul 28 17:16:06.318178 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:06.318128 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-77f648d694-zbszp" podStartSLOduration=2.360889703 podStartE2EDuration="6.318113778s" totalImagesPullingTime="3.957224075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:16:06.317033738 +0000 UTC m=+351.045535074" watchObservedRunningTime="2026-07-28 17:16:06.318113778 +0000 UTC m=+351.046615115" Jul 28 17:16:06.336548 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:06.336503 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-55557df8c-9v6bf" podStartSLOduration=2.599138037 podStartE2EDuration="6.336487146s" totalImagesPullingTime="3.737349109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:16:06.334878595 +0000 UTC m=+351.063379931" watchObservedRunningTime="2026-07-28 17:16:06.336487146 +0000 UTC m=+351.064988482" Jul 28 17:16:17.311916 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:17.311884 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-9v6bf" Jul 28 17:16:21.265247 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:21.265216 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-lnvvr" Jul 28 17:16:25.295490 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:25.295449 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-56f9798bb4-9mvl5" Jul 28 17:16:27.309205 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:16:27.309171 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-77f648d694-zbszp" Jul 28 17:17:24.285285 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:17:24.285245 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-654c557856-qkb7v"] Jul 28 17:17:24.287574 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:17:24.287551 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-654c557856-qkb7v" Jul 28 17:17:24.290600 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:17:24.290581 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-r58lg\"" Jul 28 17:17:24.291977 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:17:24.291957 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 28 17:17:24.292078 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:17:24.291985 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 28 17:17:24.292078 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:17:24.291988 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" Jul 28 17:17:24.300075 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:17:24.300056 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-654c557856-qkb7v"] Jul 28 17:17:24.387711 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:17:24.387669 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6fa97b7e-3a9a-4769-998a-40ec69ac0562-cert\") pod \"llmisvc-controller-manager-654c557856-qkb7v\" (UID: \"6fa97b7e-3a9a-4769-998a-40ec69ac0562\") " pod="kserve/llmisvc-controller-manager-654c557856-qkb7v" Jul 28 17:17:24.387909 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:17:24.387735 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tf6c\" (UniqueName: \"kubernetes.io/projected/6fa97b7e-3a9a-4769-998a-40ec69ac0562-kube-api-access-5tf6c\") pod \"llmisvc-controller-manager-654c557856-qkb7v\" (UID: \"6fa97b7e-3a9a-4769-998a-40ec69ac0562\") " pod="kserve/llmisvc-controller-manager-654c557856-qkb7v" Jul 28 17:17:24.488972 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:17:24.488931 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tf6c\" (UniqueName: \"kubernetes.io/projected/6fa97b7e-3a9a-4769-998a-40ec69ac0562-kube-api-access-5tf6c\") pod \"llmisvc-controller-manager-654c557856-qkb7v\" (UID: \"6fa97b7e-3a9a-4769-998a-40ec69ac0562\") " pod="kserve/llmisvc-controller-manager-654c557856-qkb7v" Jul 28 17:17:24.489146 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:17:24.489072 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6fa97b7e-3a9a-4769-998a-40ec69ac0562-cert\") pod \"llmisvc-controller-manager-654c557856-qkb7v\" (UID: \"6fa97b7e-3a9a-4769-998a-40ec69ac0562\") " pod="kserve/llmisvc-controller-manager-654c557856-qkb7v" Jul 28 17:17:24.491499 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:17:24.491480 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6fa97b7e-3a9a-4769-998a-40ec69ac0562-cert\") pod \"llmisvc-controller-manager-654c557856-qkb7v\" (UID: \"6fa97b7e-3a9a-4769-998a-40ec69ac0562\") " pod="kserve/llmisvc-controller-manager-654c557856-qkb7v" Jul 28 17:17:24.499755 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:17:24.499728 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tf6c\" (UniqueName: \"kubernetes.io/projected/6fa97b7e-3a9a-4769-998a-40ec69ac0562-kube-api-access-5tf6c\") pod \"llmisvc-controller-manager-654c557856-qkb7v\" (UID: \"6fa97b7e-3a9a-4769-998a-40ec69ac0562\") " pod="kserve/llmisvc-controller-manager-654c557856-qkb7v" Jul 28 17:17:24.597228 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:17:24.597132 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-654c557856-qkb7v" Jul 28 17:17:24.727245 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:17:24.727211 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-654c557856-qkb7v"] Jul 28 17:17:24.731805 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:17:24.731774 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6fa97b7e_3a9a_4769_998a_40ec69ac0562.slice/crio-7fc66a7725393de0d36a00eb1b1e9ef88504ee256f2c42f1e1895b2d4c0d6d8c WatchSource:0}: Error finding container 7fc66a7725393de0d36a00eb1b1e9ef88504ee256f2c42f1e1895b2d4c0d6d8c: Status 404 returned error can't find the container with id 7fc66a7725393de0d36a00eb1b1e9ef88504ee256f2c42f1e1895b2d4c0d6d8c Jul 28 17:17:25.625457 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:17:25.625422 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-qkb7v" event={"ID":"6fa97b7e-3a9a-4769-998a-40ec69ac0562","Type":"ContainerStarted","Data":"7fc66a7725393de0d36a00eb1b1e9ef88504ee256f2c42f1e1895b2d4c0d6d8c"} Jul 28 17:17:28.639508 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:17:28.639463 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-qkb7v" event={"ID":"6fa97b7e-3a9a-4769-998a-40ec69ac0562","Type":"ContainerStarted","Data":"79e4820e9c9123ff54750c43687a815cb83d6c2b8c693245b369b9915d862b42"} Jul 28 17:17:28.639929 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:17:28.639580 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-654c557856-qkb7v" Jul 28 17:17:28.659481 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:17:28.659420 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-654c557856-qkb7v" podStartSLOduration=1.863206847 podStartE2EDuration="4.659403447s" totalImagesPullingTime="2.7961966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:17:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:17:28.658768964 +0000 UTC m=+433.387270300" watchObservedRunningTime="2026-07-28 17:17:28.659403447 +0000 UTC m=+433.387904777" Jul 28 17:17:59.646079 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:17:59.646046 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-654c557856-qkb7v" Jul 28 17:18:26.800789 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:26.800752 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6dfb94bd9f-8tzpc"] Jul 28 17:18:26.804120 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:26.804096 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:26.819137 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:26.819110 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dfb94bd9f-8tzpc"] Jul 28 17:18:26.898300 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:26.898260 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/133915b8-a27b-40b7-b3be-5b0ad5975341-console-config\") pod \"console-6dfb94bd9f-8tzpc\" (UID: \"133915b8-a27b-40b7-b3be-5b0ad5975341\") " pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:26.898300 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:26.898314 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/133915b8-a27b-40b7-b3be-5b0ad5975341-trusted-ca-bundle\") pod \"console-6dfb94bd9f-8tzpc\" (UID: \"133915b8-a27b-40b7-b3be-5b0ad5975341\") " pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:26.898540 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:26.898338 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/133915b8-a27b-40b7-b3be-5b0ad5975341-oauth-serving-cert\") pod \"console-6dfb94bd9f-8tzpc\" (UID: \"133915b8-a27b-40b7-b3be-5b0ad5975341\") " pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:26.898540 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:26.898418 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/133915b8-a27b-40b7-b3be-5b0ad5975341-console-oauth-config\") pod \"console-6dfb94bd9f-8tzpc\" (UID: \"133915b8-a27b-40b7-b3be-5b0ad5975341\") " pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:26.898540 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:26.898448 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/133915b8-a27b-40b7-b3be-5b0ad5975341-console-serving-cert\") pod \"console-6dfb94bd9f-8tzpc\" (UID: \"133915b8-a27b-40b7-b3be-5b0ad5975341\") " pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:26.898540 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:26.898479 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/133915b8-a27b-40b7-b3be-5b0ad5975341-service-ca\") pod \"console-6dfb94bd9f-8tzpc\" (UID: \"133915b8-a27b-40b7-b3be-5b0ad5975341\") " pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:26.898540 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:26.898521 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz75v\" (UniqueName: \"kubernetes.io/projected/133915b8-a27b-40b7-b3be-5b0ad5975341-kube-api-access-gz75v\") pod \"console-6dfb94bd9f-8tzpc\" (UID: \"133915b8-a27b-40b7-b3be-5b0ad5975341\") " pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:26.999894 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:26.999838 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/133915b8-a27b-40b7-b3be-5b0ad5975341-trusted-ca-bundle\") pod \"console-6dfb94bd9f-8tzpc\" (UID: \"133915b8-a27b-40b7-b3be-5b0ad5975341\") " pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:26.999894 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:26.999903 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/133915b8-a27b-40b7-b3be-5b0ad5975341-oauth-serving-cert\") pod \"console-6dfb94bd9f-8tzpc\" (UID: \"133915b8-a27b-40b7-b3be-5b0ad5975341\") " pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:27.000156 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:26.999940 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/133915b8-a27b-40b7-b3be-5b0ad5975341-console-oauth-config\") pod \"console-6dfb94bd9f-8tzpc\" (UID: \"133915b8-a27b-40b7-b3be-5b0ad5975341\") " pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:27.000156 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:26.999967 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/133915b8-a27b-40b7-b3be-5b0ad5975341-console-serving-cert\") pod \"console-6dfb94bd9f-8tzpc\" (UID: \"133915b8-a27b-40b7-b3be-5b0ad5975341\") " pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:27.000156 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:27.000003 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/133915b8-a27b-40b7-b3be-5b0ad5975341-service-ca\") pod \"console-6dfb94bd9f-8tzpc\" (UID: \"133915b8-a27b-40b7-b3be-5b0ad5975341\") " pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:27.000156 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:27.000028 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gz75v\" (UniqueName: \"kubernetes.io/projected/133915b8-a27b-40b7-b3be-5b0ad5975341-kube-api-access-gz75v\") pod \"console-6dfb94bd9f-8tzpc\" (UID: \"133915b8-a27b-40b7-b3be-5b0ad5975341\") " pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:27.000342 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:27.000168 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/133915b8-a27b-40b7-b3be-5b0ad5975341-console-config\") pod \"console-6dfb94bd9f-8tzpc\" (UID: \"133915b8-a27b-40b7-b3be-5b0ad5975341\") " pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:27.000841 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:27.000810 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/133915b8-a27b-40b7-b3be-5b0ad5975341-oauth-serving-cert\") pod \"console-6dfb94bd9f-8tzpc\" (UID: \"133915b8-a27b-40b7-b3be-5b0ad5975341\") " pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:27.000983 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:27.000810 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/133915b8-a27b-40b7-b3be-5b0ad5975341-service-ca\") pod \"console-6dfb94bd9f-8tzpc\" (UID: \"133915b8-a27b-40b7-b3be-5b0ad5975341\") " pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:27.000983 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:27.000871 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/133915b8-a27b-40b7-b3be-5b0ad5975341-trusted-ca-bundle\") pod \"console-6dfb94bd9f-8tzpc\" (UID: \"133915b8-a27b-40b7-b3be-5b0ad5975341\") " pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:27.000983 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:27.000871 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/133915b8-a27b-40b7-b3be-5b0ad5975341-console-config\") pod \"console-6dfb94bd9f-8tzpc\" (UID: \"133915b8-a27b-40b7-b3be-5b0ad5975341\") " pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:27.002630 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:27.002599 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/133915b8-a27b-40b7-b3be-5b0ad5975341-console-serving-cert\") pod \"console-6dfb94bd9f-8tzpc\" (UID: \"133915b8-a27b-40b7-b3be-5b0ad5975341\") " pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:27.002726 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:27.002670 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/133915b8-a27b-40b7-b3be-5b0ad5975341-console-oauth-config\") pod \"console-6dfb94bd9f-8tzpc\" (UID: \"133915b8-a27b-40b7-b3be-5b0ad5975341\") " pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:27.016394 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:27.016349 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz75v\" (UniqueName: \"kubernetes.io/projected/133915b8-a27b-40b7-b3be-5b0ad5975341-kube-api-access-gz75v\") pod \"console-6dfb94bd9f-8tzpc\" (UID: \"133915b8-a27b-40b7-b3be-5b0ad5975341\") " pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:27.113919 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:27.113830 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:27.250344 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:27.250320 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6dfb94bd9f-8tzpc"] Jul 28 17:18:27.252909 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:18:27.252878 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod133915b8_a27b_40b7_b3be_5b0ad5975341.slice/crio-a76e69cc9a39c46c4db9d31278eced5a981c948009e48d05e0f4aafd210a2644 WatchSource:0}: Error finding container a76e69cc9a39c46c4db9d31278eced5a981c948009e48d05e0f4aafd210a2644: Status 404 returned error can't find the container with id a76e69cc9a39c46c4db9d31278eced5a981c948009e48d05e0f4aafd210a2644 Jul 28 17:18:27.884240 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:27.884203 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfb94bd9f-8tzpc" event={"ID":"133915b8-a27b-40b7-b3be-5b0ad5975341","Type":"ContainerStarted","Data":"37c9d4e26f65914f6d677c3b0dd65d35dcf60ca83511290cdddf113b9b7f9c5c"} Jul 28 17:18:27.884826 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:27.884254 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6dfb94bd9f-8tzpc" event={"ID":"133915b8-a27b-40b7-b3be-5b0ad5975341","Type":"ContainerStarted","Data":"a76e69cc9a39c46c4db9d31278eced5a981c948009e48d05e0f4aafd210a2644"} Jul 28 17:18:27.907342 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:27.907289 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6dfb94bd9f-8tzpc" podStartSLOduration=1.907271936 podStartE2EDuration="1.907271936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:18:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:18:27.904978604 +0000 UTC m=+492.633479941" watchObservedRunningTime="2026-07-28 17:18:27.907271936 +0000 UTC m=+492.635773272" Jul 28 17:18:37.114961 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:37.114911 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:37.115383 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:37.114982 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:37.120089 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:37.120060 2550 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:37.930499 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:37.930470 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6dfb94bd9f-8tzpc" Jul 28 17:18:37.980612 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:18:37.980584 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bc455dd8f-6ztn4"] Jul 28 17:19:03.001732 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.001668 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7bc455dd8f-6ztn4" podUID="e61b2270-bb80-4a76-a848-047004f7b43a" containerName="console" containerID="cri-o://483c376a943b4767d1468c1f831b697d3a6bbea93809514a4ff0fb21d6c25135" gracePeriod=15 Jul 28 17:19:03.249020 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.248989 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:19:03.300840 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.300800 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-trusted-ca-bundle\") pod \"e61b2270-bb80-4a76-a848-047004f7b43a\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " Jul 28 17:19:03.301019 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.300863 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-console-config\") pod \"e61b2270-bb80-4a76-a848-047004f7b43a\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " Jul 28 17:19:03.301019 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.300896 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmwvq\" (UniqueName: \"kubernetes.io/projected/e61b2270-bb80-4a76-a848-047004f7b43a-kube-api-access-pmwvq\") pod \"e61b2270-bb80-4a76-a848-047004f7b43a\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " Jul 28 17:19:03.301019 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.300913 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-service-ca\") pod \"e61b2270-bb80-4a76-a848-047004f7b43a\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " Jul 28 17:19:03.301019 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.300953 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-oauth-serving-cert\") pod \"e61b2270-bb80-4a76-a848-047004f7b43a\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " Jul 28 17:19:03.301019 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.300977 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e61b2270-bb80-4a76-a848-047004f7b43a-console-serving-cert\") pod \"e61b2270-bb80-4a76-a848-047004f7b43a\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " Jul 28 17:19:03.301243 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.301021 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e61b2270-bb80-4a76-a848-047004f7b43a-console-oauth-config\") pod \"e61b2270-bb80-4a76-a848-047004f7b43a\" (UID: \"e61b2270-bb80-4a76-a848-047004f7b43a\") " Jul 28 17:19:03.301322 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.301290 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e61b2270-bb80-4a76-a848-047004f7b43a" (UID: "e61b2270-bb80-4a76-a848-047004f7b43a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:19:03.301381 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.301332 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-console-config" (OuterVolumeSpecName: "console-config") pod "e61b2270-bb80-4a76-a848-047004f7b43a" (UID: "e61b2270-bb80-4a76-a848-047004f7b43a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:19:03.301469 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.301414 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-service-ca" (OuterVolumeSpecName: "service-ca") pod "e61b2270-bb80-4a76-a848-047004f7b43a" (UID: "e61b2270-bb80-4a76-a848-047004f7b43a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:19:03.301523 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.301482 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e61b2270-bb80-4a76-a848-047004f7b43a" (UID: "e61b2270-bb80-4a76-a848-047004f7b43a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:19:03.303188 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.303160 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e61b2270-bb80-4a76-a848-047004f7b43a-kube-api-access-pmwvq" (OuterVolumeSpecName: "kube-api-access-pmwvq") pod "e61b2270-bb80-4a76-a848-047004f7b43a" (UID: "e61b2270-bb80-4a76-a848-047004f7b43a"). InnerVolumeSpecName "kube-api-access-pmwvq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 17:19:03.303188 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.303163 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e61b2270-bb80-4a76-a848-047004f7b43a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e61b2270-bb80-4a76-a848-047004f7b43a" (UID: "e61b2270-bb80-4a76-a848-047004f7b43a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:19:03.303188 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.303181 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e61b2270-bb80-4a76-a848-047004f7b43a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e61b2270-bb80-4a76-a848-047004f7b43a" (UID: "e61b2270-bb80-4a76-a848-047004f7b43a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:19:03.402586 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.402547 2550 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-oauth-serving-cert\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:19:03.402586 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.402577 2550 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e61b2270-bb80-4a76-a848-047004f7b43a-console-serving-cert\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:19:03.402586 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.402588 2550 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e61b2270-bb80-4a76-a848-047004f7b43a-console-oauth-config\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:19:03.402586 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.402597 2550 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-trusted-ca-bundle\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:19:03.402857 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.402605 2550 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-console-config\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:19:03.402857 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.402614 2550 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmwvq\" (UniqueName: \"kubernetes.io/projected/e61b2270-bb80-4a76-a848-047004f7b43a-kube-api-access-pmwvq\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:19:03.402857 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:03.402623 2550 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e61b2270-bb80-4a76-a848-047004f7b43a-service-ca\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:19:04.032044 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:04.032009 2550 generic.go:356] "Generic (PLEG): container finished" podID="e61b2270-bb80-4a76-a848-047004f7b43a" containerID="483c376a943b4767d1468c1f831b697d3a6bbea93809514a4ff0fb21d6c25135" exitCode=0 Jul 28 17:19:04.032475 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:04.032064 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7bc455dd8f-6ztn4" Jul 28 17:19:04.032475 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:04.032064 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bc455dd8f-6ztn4" event={"ID":"e61b2270-bb80-4a76-a848-047004f7b43a","Type":"ContainerDied","Data":"483c376a943b4767d1468c1f831b697d3a6bbea93809514a4ff0fb21d6c25135"} Jul 28 17:19:04.032475 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:04.032117 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7bc455dd8f-6ztn4" event={"ID":"e61b2270-bb80-4a76-a848-047004f7b43a","Type":"ContainerDied","Data":"a8da411f54063e10b2ea87fa77f364f178a714f9bc7ca27f40508037c828d3b4"} Jul 28 17:19:04.032475 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:04.032145 2550 scope.go:117] "RemoveContainer" containerID="483c376a943b4767d1468c1f831b697d3a6bbea93809514a4ff0fb21d6c25135" Jul 28 17:19:04.040555 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:04.040531 2550 scope.go:117] "RemoveContainer" containerID="483c376a943b4767d1468c1f831b697d3a6bbea93809514a4ff0fb21d6c25135" Jul 28 17:19:04.040853 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:19:04.040836 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"483c376a943b4767d1468c1f831b697d3a6bbea93809514a4ff0fb21d6c25135\": container with ID starting with 483c376a943b4767d1468c1f831b697d3a6bbea93809514a4ff0fb21d6c25135 not found: ID does not exist" containerID="483c376a943b4767d1468c1f831b697d3a6bbea93809514a4ff0fb21d6c25135" Jul 28 17:19:04.040898 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:04.040861 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"483c376a943b4767d1468c1f831b697d3a6bbea93809514a4ff0fb21d6c25135"} err="failed to get container status \"483c376a943b4767d1468c1f831b697d3a6bbea93809514a4ff0fb21d6c25135\": rpc error: code = NotFound desc = could not find container \"483c376a943b4767d1468c1f831b697d3a6bbea93809514a4ff0fb21d6c25135\": container with ID starting with 483c376a943b4767d1468c1f831b697d3a6bbea93809514a4ff0fb21d6c25135 not found: ID does not exist" Jul 28 17:19:04.052039 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:04.052009 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7bc455dd8f-6ztn4"] Jul 28 17:19:04.056235 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:04.056208 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7bc455dd8f-6ztn4"] Jul 28 17:19:05.772352 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:19:05.772316 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e61b2270-bb80-4a76-a848-047004f7b43a" path="/var/lib/kubelet/pods/e61b2270-bb80-4a76-a848-047004f7b43a/volumes" Jul 28 17:21:56.397234 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:21:56.397200 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps"] Jul 28 17:21:56.397712 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:21:56.397533 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e61b2270-bb80-4a76-a848-047004f7b43a" containerName="console" Jul 28 17:21:56.397712 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:21:56.397544 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="e61b2270-bb80-4a76-a848-047004f7b43a" containerName="console" Jul 28 17:21:56.397712 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:21:56.397599 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="e61b2270-bb80-4a76-a848-047004f7b43a" containerName="console" Jul 28 17:21:56.400290 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:21:56.400273 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" Jul 28 17:21:56.403014 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:21:56.402993 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-d0cd1-serving-cert\"" Jul 28 17:21:56.403157 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:21:56.403126 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 28 17:21:56.403219 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:21:56.403194 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-wxvxt\"" Jul 28 17:21:56.403219 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:21:56.403196 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-d0cd1-kube-rbac-proxy-sar-config\"" Jul 28 17:21:56.409852 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:21:56.409830 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps"] Jul 28 17:21:56.425539 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:21:56.425514 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5964eb4b-a46e-4073-9637-615c451cc548-proxy-tls\") pod \"switch-graph-d0cd1-54f4d5769d-ww4ps\" (UID: \"5964eb4b-a46e-4073-9637-615c451cc548\") " pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" Jul 28 17:21:56.425633 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:21:56.425552 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5964eb4b-a46e-4073-9637-615c451cc548-openshift-service-ca-bundle\") pod \"switch-graph-d0cd1-54f4d5769d-ww4ps\" (UID: \"5964eb4b-a46e-4073-9637-615c451cc548\") " pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" Jul 28 17:21:56.526247 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:21:56.526216 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5964eb4b-a46e-4073-9637-615c451cc548-openshift-service-ca-bundle\") pod \"switch-graph-d0cd1-54f4d5769d-ww4ps\" (UID: \"5964eb4b-a46e-4073-9637-615c451cc548\") " pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" Jul 28 17:21:56.526455 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:21:56.526341 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5964eb4b-a46e-4073-9637-615c451cc548-proxy-tls\") pod \"switch-graph-d0cd1-54f4d5769d-ww4ps\" (UID: \"5964eb4b-a46e-4073-9637-615c451cc548\") " pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" Jul 28 17:21:56.526513 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:21:56.526467 2550 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-d0cd1-serving-cert: secret "switch-graph-d0cd1-serving-cert" not found Jul 28 17:21:56.526552 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:21:56.526521 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5964eb4b-a46e-4073-9637-615c451cc548-proxy-tls podName:5964eb4b-a46e-4073-9637-615c451cc548 nodeName:}" failed. No retries permitted until 2026-07-28 17:21:57.026503787 +0000 UTC m=+701.755005101 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/5964eb4b-a46e-4073-9637-615c451cc548-proxy-tls") pod "switch-graph-d0cd1-54f4d5769d-ww4ps" (UID: "5964eb4b-a46e-4073-9637-615c451cc548") : secret "switch-graph-d0cd1-serving-cert" not found Jul 28 17:21:56.526829 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:21:56.526808 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5964eb4b-a46e-4073-9637-615c451cc548-openshift-service-ca-bundle\") pod \"switch-graph-d0cd1-54f4d5769d-ww4ps\" (UID: \"5964eb4b-a46e-4073-9637-615c451cc548\") " pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" Jul 28 17:21:57.029614 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:21:57.029575 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5964eb4b-a46e-4073-9637-615c451cc548-proxy-tls\") pod \"switch-graph-d0cd1-54f4d5769d-ww4ps\" (UID: \"5964eb4b-a46e-4073-9637-615c451cc548\") " pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" Jul 28 17:21:57.032113 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:21:57.032089 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5964eb4b-a46e-4073-9637-615c451cc548-proxy-tls\") pod \"switch-graph-d0cd1-54f4d5769d-ww4ps\" (UID: \"5964eb4b-a46e-4073-9637-615c451cc548\") " pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" Jul 28 17:21:57.311121 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:21:57.311016 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" Jul 28 17:21:57.439101 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:21:57.439078 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps"] Jul 28 17:21:57.441440 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:21:57.441407 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5964eb4b_a46e_4073_9637_615c451cc548.slice/crio-013f2136bc755c720d58eb7f64bfa4660e552ba41f0f51eff5ab0d0dbf770d69 WatchSource:0}: Error finding container 013f2136bc755c720d58eb7f64bfa4660e552ba41f0f51eff5ab0d0dbf770d69: Status 404 returned error can't find the container with id 013f2136bc755c720d58eb7f64bfa4660e552ba41f0f51eff5ab0d0dbf770d69 Jul 28 17:21:57.443264 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:21:57.443238 2550 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 17:21:57.751106 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:21:57.751015 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" event={"ID":"5964eb4b-a46e-4073-9637-615c451cc548","Type":"ContainerStarted","Data":"013f2136bc755c720d58eb7f64bfa4660e552ba41f0f51eff5ab0d0dbf770d69"} Jul 28 17:22:00.766558 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:00.766517 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" event={"ID":"5964eb4b-a46e-4073-9637-615c451cc548","Type":"ContainerStarted","Data":"a3ba2fa6da20501a8c3e4fcb045a8bf68f910bb73af872bbcb86063bad196cc5"} Jul 28 17:22:00.766875 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:00.766603 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" Jul 28 17:22:00.783905 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:00.783854 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" podStartSLOduration=1.532720254 podStartE2EDuration="4.783836812s" totalImagesPullingTime="3.251116558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:21:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:22:00.783046806 +0000 UTC m=+705.511548144" watchObservedRunningTime="2026-07-28 17:22:00.783836812 +0000 UTC m=+705.512338147" Jul 28 17:22:06.777224 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:06.777192 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" Jul 28 17:22:10.648576 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:10.648495 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps"] Jul 28 17:22:10.648929 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:10.648741 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" podUID="5964eb4b-a46e-4073-9637-615c451cc548" containerName="switch-graph-d0cd1" containerID="cri-o://a3ba2fa6da20501a8c3e4fcb045a8bf68f910bb73af872bbcb86063bad196cc5" gracePeriod=30 Jul 28 17:22:11.774944 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:11.774895 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" podUID="5964eb4b-a46e-4073-9637-615c451cc548" containerName="switch-graph-d0cd1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:22:16.775979 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:16.775937 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" podUID="5964eb4b-a46e-4073-9637-615c451cc548" containerName="switch-graph-d0cd1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:22:21.774962 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:21.774922 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" podUID="5964eb4b-a46e-4073-9637-615c451cc548" containerName="switch-graph-d0cd1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:22:21.775333 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:21.775019 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" Jul 28 17:22:26.775487 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:26.775449 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" podUID="5964eb4b-a46e-4073-9637-615c451cc548" containerName="switch-graph-d0cd1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:22:31.774894 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:31.774859 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" podUID="5964eb4b-a46e-4073-9637-615c451cc548" containerName="switch-graph-d0cd1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:22:36.775339 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:36.775296 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" podUID="5964eb4b-a46e-4073-9637-615c451cc548" containerName="switch-graph-d0cd1" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:22:40.820292 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:40.820265 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" Jul 28 17:22:40.944316 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:40.944215 2550 generic.go:356] "Generic (PLEG): container finished" podID="5964eb4b-a46e-4073-9637-615c451cc548" containerID="a3ba2fa6da20501a8c3e4fcb045a8bf68f910bb73af872bbcb86063bad196cc5" exitCode=137 Jul 28 17:22:40.944316 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:40.944261 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" Jul 28 17:22:40.944316 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:40.944276 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" event={"ID":"5964eb4b-a46e-4073-9637-615c451cc548","Type":"ContainerDied","Data":"a3ba2fa6da20501a8c3e4fcb045a8bf68f910bb73af872bbcb86063bad196cc5"} Jul 28 17:22:40.944316 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:40.944319 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps" event={"ID":"5964eb4b-a46e-4073-9637-615c451cc548","Type":"ContainerDied","Data":"013f2136bc755c720d58eb7f64bfa4660e552ba41f0f51eff5ab0d0dbf770d69"} Jul 28 17:22:40.944615 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:40.944343 2550 scope.go:117] "RemoveContainer" containerID="a3ba2fa6da20501a8c3e4fcb045a8bf68f910bb73af872bbcb86063bad196cc5" Jul 28 17:22:40.953612 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:40.953593 2550 scope.go:117] "RemoveContainer" containerID="a3ba2fa6da20501a8c3e4fcb045a8bf68f910bb73af872bbcb86063bad196cc5" Jul 28 17:22:40.953954 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:22:40.953929 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3ba2fa6da20501a8c3e4fcb045a8bf68f910bb73af872bbcb86063bad196cc5\": container with ID starting with a3ba2fa6da20501a8c3e4fcb045a8bf68f910bb73af872bbcb86063bad196cc5 not found: ID does not exist" containerID="a3ba2fa6da20501a8c3e4fcb045a8bf68f910bb73af872bbcb86063bad196cc5" Jul 28 17:22:40.954024 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:40.953969 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3ba2fa6da20501a8c3e4fcb045a8bf68f910bb73af872bbcb86063bad196cc5"} err="failed to get container status \"a3ba2fa6da20501a8c3e4fcb045a8bf68f910bb73af872bbcb86063bad196cc5\": rpc error: code = NotFound desc = could not find container \"a3ba2fa6da20501a8c3e4fcb045a8bf68f910bb73af872bbcb86063bad196cc5\": container with ID starting with a3ba2fa6da20501a8c3e4fcb045a8bf68f910bb73af872bbcb86063bad196cc5 not found: ID does not exist" Jul 28 17:22:40.988277 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:40.988243 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5964eb4b-a46e-4073-9637-615c451cc548-proxy-tls\") pod \"5964eb4b-a46e-4073-9637-615c451cc548\" (UID: \"5964eb4b-a46e-4073-9637-615c451cc548\") " Jul 28 17:22:40.988486 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:40.988317 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5964eb4b-a46e-4073-9637-615c451cc548-openshift-service-ca-bundle\") pod \"5964eb4b-a46e-4073-9637-615c451cc548\" (UID: \"5964eb4b-a46e-4073-9637-615c451cc548\") " Jul 28 17:22:40.988717 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:40.988694 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5964eb4b-a46e-4073-9637-615c451cc548-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "5964eb4b-a46e-4073-9637-615c451cc548" (UID: "5964eb4b-a46e-4073-9637-615c451cc548"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:22:40.990458 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:40.990433 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5964eb4b-a46e-4073-9637-615c451cc548-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "5964eb4b-a46e-4073-9637-615c451cc548" (UID: "5964eb4b-a46e-4073-9637-615c451cc548"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:22:41.089784 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:41.089744 2550 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5964eb4b-a46e-4073-9637-615c451cc548-proxy-tls\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:22:41.089784 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:41.089777 2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5964eb4b-a46e-4073-9637-615c451cc548-openshift-service-ca-bundle\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:22:41.266228 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:41.266193 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps"] Jul 28 17:22:41.270046 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:41.270016 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-d0cd1-54f4d5769d-ww4ps"] Jul 28 17:22:41.772775 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:41.772742 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5964eb4b-a46e-4073-9637-615c451cc548" path="/var/lib/kubelet/pods/5964eb4b-a46e-4073-9637-615c451cc548/volumes" Jul 28 17:22:56.433205 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:56.433169 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-64698579b9-p8c24"] Jul 28 17:22:56.433588 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:56.433502 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5964eb4b-a46e-4073-9637-615c451cc548" containerName="switch-graph-d0cd1" Jul 28 17:22:56.433588 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:56.433515 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5964eb4b-a46e-4073-9637-615c451cc548" containerName="switch-graph-d0cd1" Jul 28 17:22:56.433588 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:56.433584 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="5964eb4b-a46e-4073-9637-615c451cc548" containerName="switch-graph-d0cd1" Jul 28 17:22:56.436896 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:56.436879 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" Jul 28 17:22:56.439965 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:56.439944 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-kube-rbac-proxy-sar-config\"" Jul 28 17:22:56.440064 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:56.440007 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-serving-cert\"" Jul 28 17:22:56.440233 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:56.440220 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-wxvxt\"" Jul 28 17:22:56.440284 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:56.440219 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 28 17:22:56.445216 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:56.445191 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-64698579b9-p8c24"] Jul 28 17:22:56.513826 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:56.513795 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cca1b114-14e7-4e18-af5e-a9b065003977-openshift-service-ca-bundle\") pod \"model-chainer-64698579b9-p8c24\" (UID: \"cca1b114-14e7-4e18-af5e-a9b065003977\") " pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" Jul 28 17:22:56.513826 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:56.513828 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cca1b114-14e7-4e18-af5e-a9b065003977-proxy-tls\") pod \"model-chainer-64698579b9-p8c24\" (UID: \"cca1b114-14e7-4e18-af5e-a9b065003977\") " pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" Jul 28 17:22:56.614930 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:56.614899 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cca1b114-14e7-4e18-af5e-a9b065003977-openshift-service-ca-bundle\") pod \"model-chainer-64698579b9-p8c24\" (UID: \"cca1b114-14e7-4e18-af5e-a9b065003977\") " pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" Jul 28 17:22:56.614930 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:56.614934 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cca1b114-14e7-4e18-af5e-a9b065003977-proxy-tls\") pod \"model-chainer-64698579b9-p8c24\" (UID: \"cca1b114-14e7-4e18-af5e-a9b065003977\") " pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" Jul 28 17:22:56.615153 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:22:56.615059 2550 secret.go:189] Couldn't get secret kserve-ci-e2e-test/model-chainer-serving-cert: secret "model-chainer-serving-cert" not found Jul 28 17:22:56.615153 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:22:56.615123 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cca1b114-14e7-4e18-af5e-a9b065003977-proxy-tls podName:cca1b114-14e7-4e18-af5e-a9b065003977 nodeName:}" failed. No retries permitted until 2026-07-28 17:22:57.115104766 +0000 UTC m=+761.843606079 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/cca1b114-14e7-4e18-af5e-a9b065003977-proxy-tls") pod "model-chainer-64698579b9-p8c24" (UID: "cca1b114-14e7-4e18-af5e-a9b065003977") : secret "model-chainer-serving-cert" not found Jul 28 17:22:56.615623 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:56.615603 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cca1b114-14e7-4e18-af5e-a9b065003977-openshift-service-ca-bundle\") pod \"model-chainer-64698579b9-p8c24\" (UID: \"cca1b114-14e7-4e18-af5e-a9b065003977\") " pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" Jul 28 17:22:57.119599 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:57.119565 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cca1b114-14e7-4e18-af5e-a9b065003977-proxy-tls\") pod \"model-chainer-64698579b9-p8c24\" (UID: \"cca1b114-14e7-4e18-af5e-a9b065003977\") " pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" Jul 28 17:22:57.121961 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:57.121937 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cca1b114-14e7-4e18-af5e-a9b065003977-proxy-tls\") pod \"model-chainer-64698579b9-p8c24\" (UID: \"cca1b114-14e7-4e18-af5e-a9b065003977\") " pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" Jul 28 17:22:57.347910 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:57.347864 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" Jul 28 17:22:57.478208 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:57.478176 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-64698579b9-p8c24"] Jul 28 17:22:57.481513 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:22:57.481484 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcca1b114_14e7_4e18_af5e_a9b065003977.slice/crio-ff43d99c2afbd0cc9bb45f46b3d1dd1330c3e1c571d67935a1d3a8eea519d85e WatchSource:0}: Error finding container ff43d99c2afbd0cc9bb45f46b3d1dd1330c3e1c571d67935a1d3a8eea519d85e: Status 404 returned error can't find the container with id ff43d99c2afbd0cc9bb45f46b3d1dd1330c3e1c571d67935a1d3a8eea519d85e Jul 28 17:22:58.014752 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:58.014715 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" event={"ID":"cca1b114-14e7-4e18-af5e-a9b065003977","Type":"ContainerStarted","Data":"4ee27749721e8d468784c05527db1e70ceaa99995b4d9c19d4bbc791af8fdc9d"} Jul 28 17:22:58.014935 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:58.014768 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" event={"ID":"cca1b114-14e7-4e18-af5e-a9b065003977","Type":"ContainerStarted","Data":"ff43d99c2afbd0cc9bb45f46b3d1dd1330c3e1c571d67935a1d3a8eea519d85e"} Jul 28 17:22:58.014935 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:58.014802 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" Jul 28 17:22:58.033271 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:22:58.033217 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" podStartSLOduration=2.033201438 podStartE2EDuration="2.033201438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:22:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:22:58.032877088 +0000 UTC m=+762.761378423" watchObservedRunningTime="2026-07-28 17:22:58.033201438 +0000 UTC m=+762.761702774" Jul 28 17:23:04.024679 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:04.024649 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" Jul 28 17:23:06.540990 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:06.540955 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-64698579b9-p8c24"] Jul 28 17:23:06.541375 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:06.541131 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" podUID="cca1b114-14e7-4e18-af5e-a9b065003977" containerName="model-chainer" containerID="cri-o://4ee27749721e8d468784c05527db1e70ceaa99995b4d9c19d4bbc791af8fdc9d" gracePeriod=30 Jul 28 17:23:09.022716 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:09.022670 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" podUID="cca1b114-14e7-4e18-af5e-a9b065003977" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:23:10.906658 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:10.906624 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg"] Jul 28 17:23:10.910270 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:10.910246 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" Jul 28 17:23:10.912934 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:10.912908 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-1fbfc-serving-cert\"" Jul 28 17:23:10.913054 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:10.912908 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-1fbfc-kube-rbac-proxy-sar-config\"" Jul 28 17:23:10.919873 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:10.919848 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg"] Jul 28 17:23:11.023425 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:11.023375 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86aed8ef-acff-44e0-b41b-2d88a4663691-openshift-service-ca-bundle\") pod \"switch-graph-1fbfc-6cb7c7477-hpmmg\" (UID: \"86aed8ef-acff-44e0-b41b-2d88a4663691\") " pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" Jul 28 17:23:11.023601 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:11.023442 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86aed8ef-acff-44e0-b41b-2d88a4663691-proxy-tls\") pod \"switch-graph-1fbfc-6cb7c7477-hpmmg\" (UID: \"86aed8ef-acff-44e0-b41b-2d88a4663691\") " pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" Jul 28 17:23:11.124892 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:11.124854 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86aed8ef-acff-44e0-b41b-2d88a4663691-openshift-service-ca-bundle\") pod \"switch-graph-1fbfc-6cb7c7477-hpmmg\" (UID: \"86aed8ef-acff-44e0-b41b-2d88a4663691\") " pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" Jul 28 17:23:11.124892 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:11.124898 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86aed8ef-acff-44e0-b41b-2d88a4663691-proxy-tls\") pod \"switch-graph-1fbfc-6cb7c7477-hpmmg\" (UID: \"86aed8ef-acff-44e0-b41b-2d88a4663691\") " pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" Jul 28 17:23:11.125621 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:11.125595 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86aed8ef-acff-44e0-b41b-2d88a4663691-openshift-service-ca-bundle\") pod \"switch-graph-1fbfc-6cb7c7477-hpmmg\" (UID: \"86aed8ef-acff-44e0-b41b-2d88a4663691\") " pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" Jul 28 17:23:11.127401 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:11.127368 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86aed8ef-acff-44e0-b41b-2d88a4663691-proxy-tls\") pod \"switch-graph-1fbfc-6cb7c7477-hpmmg\" (UID: \"86aed8ef-acff-44e0-b41b-2d88a4663691\") " pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" Jul 28 17:23:11.221636 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:11.221556 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" Jul 28 17:23:11.350607 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:11.350581 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg"] Jul 28 17:23:11.352681 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:23:11.352654 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86aed8ef_acff_44e0_b41b_2d88a4663691.slice/crio-a924baa2a448c9a30691930797798976d07aa06d596a627f9fd029c6bd270b70 WatchSource:0}: Error finding container a924baa2a448c9a30691930797798976d07aa06d596a627f9fd029c6bd270b70: Status 404 returned error can't find the container with id a924baa2a448c9a30691930797798976d07aa06d596a627f9fd029c6bd270b70 Jul 28 17:23:12.074418 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:12.074365 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" event={"ID":"86aed8ef-acff-44e0-b41b-2d88a4663691","Type":"ContainerStarted","Data":"352137fd4b07504e12d4ec1acd28c76182726431c1d9c1598d176bed32db2589"} Jul 28 17:23:12.074802 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:12.074439 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" event={"ID":"86aed8ef-acff-44e0-b41b-2d88a4663691","Type":"ContainerStarted","Data":"a924baa2a448c9a30691930797798976d07aa06d596a627f9fd029c6bd270b70"} Jul 28 17:23:12.074802 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:12.074480 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" Jul 28 17:23:12.093085 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:12.093036 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" podStartSLOduration=2.093019664 podStartE2EDuration="2.093019664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:23:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:23:12.091203177 +0000 UTC m=+776.819704513" watchObservedRunningTime="2026-07-28 17:23:12.093019664 +0000 UTC m=+776.821520999" Jul 28 17:23:14.023059 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:14.023021 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" podUID="cca1b114-14e7-4e18-af5e-a9b065003977" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:23:18.084610 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:18.084579 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" Jul 28 17:23:19.022862 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:19.022814 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" podUID="cca1b114-14e7-4e18-af5e-a9b065003977" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:23:19.023037 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:19.022921 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" Jul 28 17:23:24.022820 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:24.022775 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" podUID="cca1b114-14e7-4e18-af5e-a9b065003977" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:23:29.023347 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:29.023302 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" podUID="cca1b114-14e7-4e18-af5e-a9b065003977" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:23:34.022983 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:34.022941 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" podUID="cca1b114-14e7-4e18-af5e-a9b065003977" containerName="model-chainer" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:23:36.680668 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:36.680639 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" Jul 28 17:23:36.841709 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:36.841609 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cca1b114-14e7-4e18-af5e-a9b065003977-proxy-tls\") pod \"cca1b114-14e7-4e18-af5e-a9b065003977\" (UID: \"cca1b114-14e7-4e18-af5e-a9b065003977\") " Jul 28 17:23:36.841709 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:36.841687 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cca1b114-14e7-4e18-af5e-a9b065003977-openshift-service-ca-bundle\") pod \"cca1b114-14e7-4e18-af5e-a9b065003977\" (UID: \"cca1b114-14e7-4e18-af5e-a9b065003977\") " Jul 28 17:23:36.842031 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:36.842005 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cca1b114-14e7-4e18-af5e-a9b065003977-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "cca1b114-14e7-4e18-af5e-a9b065003977" (UID: "cca1b114-14e7-4e18-af5e-a9b065003977"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:23:36.843678 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:36.843655 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cca1b114-14e7-4e18-af5e-a9b065003977-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "cca1b114-14e7-4e18-af5e-a9b065003977" (UID: "cca1b114-14e7-4e18-af5e-a9b065003977"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:23:36.942907 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:36.942864 2550 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cca1b114-14e7-4e18-af5e-a9b065003977-proxy-tls\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:23:36.942907 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:36.942898 2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cca1b114-14e7-4e18-af5e-a9b065003977-openshift-service-ca-bundle\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:23:37.181108 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:37.181024 2550 generic.go:356] "Generic (PLEG): container finished" podID="cca1b114-14e7-4e18-af5e-a9b065003977" containerID="4ee27749721e8d468784c05527db1e70ceaa99995b4d9c19d4bbc791af8fdc9d" exitCode=0 Jul 28 17:23:37.181108 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:37.181064 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" Jul 28 17:23:37.181283 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:37.181113 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" event={"ID":"cca1b114-14e7-4e18-af5e-a9b065003977","Type":"ContainerDied","Data":"4ee27749721e8d468784c05527db1e70ceaa99995b4d9c19d4bbc791af8fdc9d"} Jul 28 17:23:37.181283 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:37.181161 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-64698579b9-p8c24" event={"ID":"cca1b114-14e7-4e18-af5e-a9b065003977","Type":"ContainerDied","Data":"ff43d99c2afbd0cc9bb45f46b3d1dd1330c3e1c571d67935a1d3a8eea519d85e"} Jul 28 17:23:37.181283 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:37.181182 2550 scope.go:117] "RemoveContainer" containerID="4ee27749721e8d468784c05527db1e70ceaa99995b4d9c19d4bbc791af8fdc9d" Jul 28 17:23:37.189880 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:37.189863 2550 scope.go:117] "RemoveContainer" containerID="4ee27749721e8d468784c05527db1e70ceaa99995b4d9c19d4bbc791af8fdc9d" Jul 28 17:23:37.190103 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:23:37.190086 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ee27749721e8d468784c05527db1e70ceaa99995b4d9c19d4bbc791af8fdc9d\": container with ID starting with 4ee27749721e8d468784c05527db1e70ceaa99995b4d9c19d4bbc791af8fdc9d not found: ID does not exist" containerID="4ee27749721e8d468784c05527db1e70ceaa99995b4d9c19d4bbc791af8fdc9d" Jul 28 17:23:37.190167 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:37.190112 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ee27749721e8d468784c05527db1e70ceaa99995b4d9c19d4bbc791af8fdc9d"} err="failed to get container status \"4ee27749721e8d468784c05527db1e70ceaa99995b4d9c19d4bbc791af8fdc9d\": rpc error: code = NotFound desc = could not find container \"4ee27749721e8d468784c05527db1e70ceaa99995b4d9c19d4bbc791af8fdc9d\": container with ID starting with 4ee27749721e8d468784c05527db1e70ceaa99995b4d9c19d4bbc791af8fdc9d not found: ID does not exist" Jul 28 17:23:37.203098 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:37.203075 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-64698579b9-p8c24"] Jul 28 17:23:37.207961 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:37.207941 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-64698579b9-p8c24"] Jul 28 17:23:37.771802 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:23:37.771771 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cca1b114-14e7-4e18-af5e-a9b065003977" path="/var/lib/kubelet/pods/cca1b114-14e7-4e18-af5e-a9b065003977/volumes" Jul 28 17:24:06.801796 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:06.801758 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl"] Jul 28 17:24:06.802190 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:06.802103 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cca1b114-14e7-4e18-af5e-a9b065003977" containerName="model-chainer" Jul 28 17:24:06.802190 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:06.802118 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="cca1b114-14e7-4e18-af5e-a9b065003977" containerName="model-chainer" Jul 28 17:24:06.802190 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:06.802173 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="cca1b114-14e7-4e18-af5e-a9b065003977" containerName="model-chainer" Jul 28 17:24:06.805379 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:06.805362 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" Jul 28 17:24:06.808450 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:06.808424 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-3a5ae-serving-cert\"" Jul 28 17:24:06.808668 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:06.808650 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-3a5ae-kube-rbac-proxy-sar-config\"" Jul 28 17:24:06.827671 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:06.827634 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl"] Jul 28 17:24:06.890094 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:06.890057 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acd97157-d6c7-49b7-b431-5b9be226c95a-openshift-service-ca-bundle\") pod \"sequence-graph-3a5ae-6966f7f449-bgxjl\" (UID: \"acd97157-d6c7-49b7-b431-5b9be226c95a\") " pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" Jul 28 17:24:06.890283 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:06.890124 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/acd97157-d6c7-49b7-b431-5b9be226c95a-proxy-tls\") pod \"sequence-graph-3a5ae-6966f7f449-bgxjl\" (UID: \"acd97157-d6c7-49b7-b431-5b9be226c95a\") " pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" Jul 28 17:24:06.991567 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:06.991528 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/acd97157-d6c7-49b7-b431-5b9be226c95a-proxy-tls\") pod \"sequence-graph-3a5ae-6966f7f449-bgxjl\" (UID: \"acd97157-d6c7-49b7-b431-5b9be226c95a\") " pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" Jul 28 17:24:06.991760 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:06.991600 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acd97157-d6c7-49b7-b431-5b9be226c95a-openshift-service-ca-bundle\") pod \"sequence-graph-3a5ae-6966f7f449-bgxjl\" (UID: \"acd97157-d6c7-49b7-b431-5b9be226c95a\") " pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" Jul 28 17:24:06.991760 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:24:06.991685 2550 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-3a5ae-serving-cert: secret "sequence-graph-3a5ae-serving-cert" not found Jul 28 17:24:06.991760 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:24:06.991752 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acd97157-d6c7-49b7-b431-5b9be226c95a-proxy-tls podName:acd97157-d6c7-49b7-b431-5b9be226c95a nodeName:}" failed. No retries permitted until 2026-07-28 17:24:07.491735845 +0000 UTC m=+832.220237159 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/acd97157-d6c7-49b7-b431-5b9be226c95a-proxy-tls") pod "sequence-graph-3a5ae-6966f7f449-bgxjl" (UID: "acd97157-d6c7-49b7-b431-5b9be226c95a") : secret "sequence-graph-3a5ae-serving-cert" not found Jul 28 17:24:06.992230 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:06.992211 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acd97157-d6c7-49b7-b431-5b9be226c95a-openshift-service-ca-bundle\") pod \"sequence-graph-3a5ae-6966f7f449-bgxjl\" (UID: \"acd97157-d6c7-49b7-b431-5b9be226c95a\") " pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" Jul 28 17:24:07.495957 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:07.495915 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/acd97157-d6c7-49b7-b431-5b9be226c95a-proxy-tls\") pod \"sequence-graph-3a5ae-6966f7f449-bgxjl\" (UID: \"acd97157-d6c7-49b7-b431-5b9be226c95a\") " pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" Jul 28 17:24:07.498398 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:07.498367 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/acd97157-d6c7-49b7-b431-5b9be226c95a-proxy-tls\") pod \"sequence-graph-3a5ae-6966f7f449-bgxjl\" (UID: \"acd97157-d6c7-49b7-b431-5b9be226c95a\") " pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" Jul 28 17:24:07.715284 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:07.715237 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" Jul 28 17:24:07.857829 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:07.857791 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl"] Jul 28 17:24:07.860621 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:24:07.860594 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacd97157_d6c7_49b7_b431_5b9be226c95a.slice/crio-c1fb54736f0f892d0d53339fe9b3068234ab497c1f41e30d664e3116c184022b WatchSource:0}: Error finding container c1fb54736f0f892d0d53339fe9b3068234ab497c1f41e30d664e3116c184022b: Status 404 returned error can't find the container with id c1fb54736f0f892d0d53339fe9b3068234ab497c1f41e30d664e3116c184022b Jul 28 17:24:08.319342 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:08.319308 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" event={"ID":"acd97157-d6c7-49b7-b431-5b9be226c95a","Type":"ContainerStarted","Data":"b4b6ef4931d2a31b8a641cbafcc4a80ba7cf1958de47614845e97c3b67045a85"} Jul 28 17:24:08.319532 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:08.319363 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" event={"ID":"acd97157-d6c7-49b7-b431-5b9be226c95a","Type":"ContainerStarted","Data":"c1fb54736f0f892d0d53339fe9b3068234ab497c1f41e30d664e3116c184022b"} Jul 28 17:24:08.319532 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:08.319415 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" Jul 28 17:24:08.340624 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:08.340568 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" podStartSLOduration=2.340552926 podStartE2EDuration="2.340552926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:24:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:24:08.339032145 +0000 UTC m=+833.067533481" watchObservedRunningTime="2026-07-28 17:24:08.340552926 +0000 UTC m=+833.069054262" Jul 28 17:24:14.328281 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:24:14.328249 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" Jul 28 17:31:25.629654 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:25.629618 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg"] Jul 28 17:31:25.631947 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:25.629863 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" podUID="86aed8ef-acff-44e0-b41b-2d88a4663691" containerName="switch-graph-1fbfc" containerID="cri-o://352137fd4b07504e12d4ec1acd28c76182726431c1d9c1598d176bed32db2589" gracePeriod=30 Jul 28 17:31:28.082535 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:28.082494 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" podUID="86aed8ef-acff-44e0-b41b-2d88a4663691" containerName="switch-graph-1fbfc" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:31:33.083346 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:33.083307 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" podUID="86aed8ef-acff-44e0-b41b-2d88a4663691" containerName="switch-graph-1fbfc" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:31:38.082935 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:38.082893 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" podUID="86aed8ef-acff-44e0-b41b-2d88a4663691" containerName="switch-graph-1fbfc" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:31:38.083319 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:38.083011 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" Jul 28 17:31:43.083475 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:43.083428 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" podUID="86aed8ef-acff-44e0-b41b-2d88a4663691" containerName="switch-graph-1fbfc" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:31:48.082529 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:48.082488 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" podUID="86aed8ef-acff-44e0-b41b-2d88a4663691" containerName="switch-graph-1fbfc" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:31:53.082983 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:53.082938 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" podUID="86aed8ef-acff-44e0-b41b-2d88a4663691" containerName="switch-graph-1fbfc" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:31:55.771295 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:55.771269 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" Jul 28 17:31:55.899735 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:55.899636 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86aed8ef-acff-44e0-b41b-2d88a4663691-proxy-tls\") pod \"86aed8ef-acff-44e0-b41b-2d88a4663691\" (UID: \"86aed8ef-acff-44e0-b41b-2d88a4663691\") " Jul 28 17:31:55.899735 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:55.899708 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86aed8ef-acff-44e0-b41b-2d88a4663691-openshift-service-ca-bundle\") pod \"86aed8ef-acff-44e0-b41b-2d88a4663691\" (UID: \"86aed8ef-acff-44e0-b41b-2d88a4663691\") " Jul 28 17:31:55.900083 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:55.900060 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86aed8ef-acff-44e0-b41b-2d88a4663691-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "86aed8ef-acff-44e0-b41b-2d88a4663691" (UID: "86aed8ef-acff-44e0-b41b-2d88a4663691"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:31:55.901753 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:55.901723 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86aed8ef-acff-44e0-b41b-2d88a4663691-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "86aed8ef-acff-44e0-b41b-2d88a4663691" (UID: "86aed8ef-acff-44e0-b41b-2d88a4663691"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:31:56.001056 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:56.001004 2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/86aed8ef-acff-44e0-b41b-2d88a4663691-openshift-service-ca-bundle\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:31:56.001056 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:56.001050 2550 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/86aed8ef-acff-44e0-b41b-2d88a4663691-proxy-tls\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:31:56.297016 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:56.296981 2550 generic.go:356] "Generic (PLEG): container finished" podID="86aed8ef-acff-44e0-b41b-2d88a4663691" containerID="352137fd4b07504e12d4ec1acd28c76182726431c1d9c1598d176bed32db2589" exitCode=0 Jul 28 17:31:56.297197 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:56.297044 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" Jul 28 17:31:56.297197 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:56.297064 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" event={"ID":"86aed8ef-acff-44e0-b41b-2d88a4663691","Type":"ContainerDied","Data":"352137fd4b07504e12d4ec1acd28c76182726431c1d9c1598d176bed32db2589"} Jul 28 17:31:56.297197 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:56.297160 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg" event={"ID":"86aed8ef-acff-44e0-b41b-2d88a4663691","Type":"ContainerDied","Data":"a924baa2a448c9a30691930797798976d07aa06d596a627f9fd029c6bd270b70"} Jul 28 17:31:56.297197 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:56.297189 2550 scope.go:117] "RemoveContainer" containerID="352137fd4b07504e12d4ec1acd28c76182726431c1d9c1598d176bed32db2589" Jul 28 17:31:56.305888 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:56.305869 2550 scope.go:117] "RemoveContainer" containerID="352137fd4b07504e12d4ec1acd28c76182726431c1d9c1598d176bed32db2589" Jul 28 17:31:56.306168 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:31:56.306145 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"352137fd4b07504e12d4ec1acd28c76182726431c1d9c1598d176bed32db2589\": container with ID starting with 352137fd4b07504e12d4ec1acd28c76182726431c1d9c1598d176bed32db2589 not found: ID does not exist" containerID="352137fd4b07504e12d4ec1acd28c76182726431c1d9c1598d176bed32db2589" Jul 28 17:31:56.306268 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:56.306179 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"352137fd4b07504e12d4ec1acd28c76182726431c1d9c1598d176bed32db2589"} err="failed to get container status \"352137fd4b07504e12d4ec1acd28c76182726431c1d9c1598d176bed32db2589\": rpc error: code = NotFound desc = could not find container \"352137fd4b07504e12d4ec1acd28c76182726431c1d9c1598d176bed32db2589\": container with ID starting with 352137fd4b07504e12d4ec1acd28c76182726431c1d9c1598d176bed32db2589 not found: ID does not exist" Jul 28 17:31:56.318230 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:56.318196 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg"] Jul 28 17:31:56.322173 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:56.322144 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-1fbfc-6cb7c7477-hpmmg"] Jul 28 17:31:57.772094 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:31:57.772057 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86aed8ef-acff-44e0-b41b-2d88a4663691" path="/var/lib/kubelet/pods/86aed8ef-acff-44e0-b41b-2d88a4663691/volumes" Jul 28 17:32:21.359351 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:21.359316 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl"] Jul 28 17:32:21.359784 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:21.359533 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" podUID="acd97157-d6c7-49b7-b431-5b9be226c95a" containerName="sequence-graph-3a5ae" containerID="cri-o://b4b6ef4931d2a31b8a641cbafcc4a80ba7cf1958de47614845e97c3b67045a85" gracePeriod=30 Jul 28 17:32:24.327675 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:24.327629 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" podUID="acd97157-d6c7-49b7-b431-5b9be226c95a" containerName="sequence-graph-3a5ae" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:32:25.867352 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:25.867321 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7"] Jul 28 17:32:25.867765 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:25.867649 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86aed8ef-acff-44e0-b41b-2d88a4663691" containerName="switch-graph-1fbfc" Jul 28 17:32:25.867765 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:25.867662 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="86aed8ef-acff-44e0-b41b-2d88a4663691" containerName="switch-graph-1fbfc" Jul 28 17:32:25.867765 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:25.867725 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="86aed8ef-acff-44e0-b41b-2d88a4663691" containerName="switch-graph-1fbfc" Jul 28 17:32:25.871665 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:25.871647 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" Jul 28 17:32:25.874289 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:25.874260 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-b0164-serving-cert\"" Jul 28 17:32:25.874289 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:25.874265 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-b0164-kube-rbac-proxy-sar-config\"" Jul 28 17:32:25.880200 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:25.880176 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7"] Jul 28 17:32:25.941936 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:25.941901 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b9cf40c6-297f-4f91-b813-1f04685df5e0-proxy-tls\") pod \"ensemble-graph-b0164-855bf76f5c-9lgt7\" (UID: \"b9cf40c6-297f-4f91-b813-1f04685df5e0\") " pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" Jul 28 17:32:25.942152 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:25.941955 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9cf40c6-297f-4f91-b813-1f04685df5e0-openshift-service-ca-bundle\") pod \"ensemble-graph-b0164-855bf76f5c-9lgt7\" (UID: \"b9cf40c6-297f-4f91-b813-1f04685df5e0\") " pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" Jul 28 17:32:26.043159 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:26.043115 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b9cf40c6-297f-4f91-b813-1f04685df5e0-proxy-tls\") pod \"ensemble-graph-b0164-855bf76f5c-9lgt7\" (UID: \"b9cf40c6-297f-4f91-b813-1f04685df5e0\") " pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" Jul 28 17:32:26.043350 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:26.043177 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9cf40c6-297f-4f91-b813-1f04685df5e0-openshift-service-ca-bundle\") pod \"ensemble-graph-b0164-855bf76f5c-9lgt7\" (UID: \"b9cf40c6-297f-4f91-b813-1f04685df5e0\") " pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" Jul 28 17:32:26.043814 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:26.043794 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9cf40c6-297f-4f91-b813-1f04685df5e0-openshift-service-ca-bundle\") pod \"ensemble-graph-b0164-855bf76f5c-9lgt7\" (UID: \"b9cf40c6-297f-4f91-b813-1f04685df5e0\") " pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" Jul 28 17:32:26.045657 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:26.045635 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b9cf40c6-297f-4f91-b813-1f04685df5e0-proxy-tls\") pod \"ensemble-graph-b0164-855bf76f5c-9lgt7\" (UID: \"b9cf40c6-297f-4f91-b813-1f04685df5e0\") " pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" Jul 28 17:32:26.182303 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:26.182204 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" Jul 28 17:32:26.311304 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:26.311272 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7"] Jul 28 17:32:26.314160 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:32:26.314133 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9cf40c6_297f_4f91_b813_1f04685df5e0.slice/crio-cdbb5c24927d1236cd8a357683876ad8dd7f0deb692db36f33efc64816998a9a WatchSource:0}: Error finding container cdbb5c24927d1236cd8a357683876ad8dd7f0deb692db36f33efc64816998a9a: Status 404 returned error can't find the container with id cdbb5c24927d1236cd8a357683876ad8dd7f0deb692db36f33efc64816998a9a Jul 28 17:32:26.431438 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:26.431401 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" event={"ID":"b9cf40c6-297f-4f91-b813-1f04685df5e0","Type":"ContainerStarted","Data":"44bfc5615778e143c51e4efe0753aa9bd88757b048f283b2d9a1cfcf58bc1f60"} Jul 28 17:32:26.431617 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:26.431465 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" event={"ID":"b9cf40c6-297f-4f91-b813-1f04685df5e0","Type":"ContainerStarted","Data":"cdbb5c24927d1236cd8a357683876ad8dd7f0deb692db36f33efc64816998a9a"} Jul 28 17:32:26.431617 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:26.431492 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" Jul 28 17:32:26.448233 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:26.448131 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" podStartSLOduration=1.448116384 podStartE2EDuration="1.448116384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:32:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:32:26.447921023 +0000 UTC m=+1331.176422353" watchObservedRunningTime="2026-07-28 17:32:26.448116384 +0000 UTC m=+1331.176617719" Jul 28 17:32:29.327361 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:29.327319 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" podUID="acd97157-d6c7-49b7-b431-5b9be226c95a" containerName="sequence-graph-3a5ae" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:32:32.442270 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:32.442235 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" Jul 28 17:32:34.327183 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:34.327144 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" podUID="acd97157-d6c7-49b7-b431-5b9be226c95a" containerName="sequence-graph-3a5ae" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:32:34.327605 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:34.327252 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" Jul 28 17:32:35.941149 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:35.941063 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7"] Jul 28 17:32:35.941522 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:35.941264 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" podUID="b9cf40c6-297f-4f91-b813-1f04685df5e0" containerName="ensemble-graph-b0164" containerID="cri-o://44bfc5615778e143c51e4efe0753aa9bd88757b048f283b2d9a1cfcf58bc1f60" gracePeriod=30 Jul 28 17:32:37.439429 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:37.439360 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" podUID="b9cf40c6-297f-4f91-b813-1f04685df5e0" containerName="ensemble-graph-b0164" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:32:39.327375 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:39.327285 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" podUID="acd97157-d6c7-49b7-b431-5b9be226c95a" containerName="sequence-graph-3a5ae" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:32:42.439320 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:42.439276 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" podUID="b9cf40c6-297f-4f91-b813-1f04685df5e0" containerName="ensemble-graph-b0164" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:32:44.327074 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:44.327027 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" podUID="acd97157-d6c7-49b7-b431-5b9be226c95a" containerName="sequence-graph-3a5ae" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:32:47.439611 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:47.439564 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" podUID="b9cf40c6-297f-4f91-b813-1f04685df5e0" containerName="ensemble-graph-b0164" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:32:47.440163 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:47.439682 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" Jul 28 17:32:49.327310 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:49.327275 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" podUID="acd97157-d6c7-49b7-b431-5b9be226c95a" containerName="sequence-graph-3a5ae" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:32:51.505071 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:51.505044 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" Jul 28 17:32:51.542463 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:51.542428 2550 generic.go:356] "Generic (PLEG): container finished" podID="acd97157-d6c7-49b7-b431-5b9be226c95a" containerID="b4b6ef4931d2a31b8a641cbafcc4a80ba7cf1958de47614845e97c3b67045a85" exitCode=0 Jul 28 17:32:51.542642 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:51.542473 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" Jul 28 17:32:51.542642 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:51.542512 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" event={"ID":"acd97157-d6c7-49b7-b431-5b9be226c95a","Type":"ContainerDied","Data":"b4b6ef4931d2a31b8a641cbafcc4a80ba7cf1958de47614845e97c3b67045a85"} Jul 28 17:32:51.542642 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:51.542598 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl" event={"ID":"acd97157-d6c7-49b7-b431-5b9be226c95a","Type":"ContainerDied","Data":"c1fb54736f0f892d0d53339fe9b3068234ab497c1f41e30d664e3116c184022b"} Jul 28 17:32:51.542642 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:51.542635 2550 scope.go:117] "RemoveContainer" containerID="b4b6ef4931d2a31b8a641cbafcc4a80ba7cf1958de47614845e97c3b67045a85" Jul 28 17:32:51.550766 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:51.550751 2550 scope.go:117] "RemoveContainer" containerID="b4b6ef4931d2a31b8a641cbafcc4a80ba7cf1958de47614845e97c3b67045a85" Jul 28 17:32:51.551025 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:32:51.551006 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4b6ef4931d2a31b8a641cbafcc4a80ba7cf1958de47614845e97c3b67045a85\": container with ID starting with b4b6ef4931d2a31b8a641cbafcc4a80ba7cf1958de47614845e97c3b67045a85 not found: ID does not exist" containerID="b4b6ef4931d2a31b8a641cbafcc4a80ba7cf1958de47614845e97c3b67045a85" Jul 28 17:32:51.551091 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:51.551032 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4b6ef4931d2a31b8a641cbafcc4a80ba7cf1958de47614845e97c3b67045a85"} err="failed to get container status \"b4b6ef4931d2a31b8a641cbafcc4a80ba7cf1958de47614845e97c3b67045a85\": rpc error: code = NotFound desc = could not find container \"b4b6ef4931d2a31b8a641cbafcc4a80ba7cf1958de47614845e97c3b67045a85\": container with ID starting with b4b6ef4931d2a31b8a641cbafcc4a80ba7cf1958de47614845e97c3b67045a85 not found: ID does not exist" Jul 28 17:32:51.551217 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:51.551206 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acd97157-d6c7-49b7-b431-5b9be226c95a-openshift-service-ca-bundle\") pod \"acd97157-d6c7-49b7-b431-5b9be226c95a\" (UID: \"acd97157-d6c7-49b7-b431-5b9be226c95a\") " Jul 28 17:32:51.551295 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:51.551283 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/acd97157-d6c7-49b7-b431-5b9be226c95a-proxy-tls\") pod \"acd97157-d6c7-49b7-b431-5b9be226c95a\" (UID: \"acd97157-d6c7-49b7-b431-5b9be226c95a\") " Jul 28 17:32:51.551587 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:51.551557 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acd97157-d6c7-49b7-b431-5b9be226c95a-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "acd97157-d6c7-49b7-b431-5b9be226c95a" (UID: "acd97157-d6c7-49b7-b431-5b9be226c95a"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:32:51.553236 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:51.553207 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acd97157-d6c7-49b7-b431-5b9be226c95a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "acd97157-d6c7-49b7-b431-5b9be226c95a" (UID: "acd97157-d6c7-49b7-b431-5b9be226c95a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:32:51.652710 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:51.652615 2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acd97157-d6c7-49b7-b431-5b9be226c95a-openshift-service-ca-bundle\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:32:51.652710 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:51.652649 2550 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/acd97157-d6c7-49b7-b431-5b9be226c95a-proxy-tls\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:32:51.859121 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:51.859079 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl"] Jul 28 17:32:51.864914 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:51.864884 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-3a5ae-6966f7f449-bgxjl"] Jul 28 17:32:52.439119 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:52.439077 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" podUID="b9cf40c6-297f-4f91-b813-1f04685df5e0" containerName="ensemble-graph-b0164" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:32:53.771560 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:53.771528 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="acd97157-d6c7-49b7-b431-5b9be226c95a" path="/var/lib/kubelet/pods/acd97157-d6c7-49b7-b431-5b9be226c95a/volumes" Jul 28 17:32:57.439940 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:32:57.439895 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" podUID="b9cf40c6-297f-4f91-b813-1f04685df5e0" containerName="ensemble-graph-b0164" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:33:02.439775 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:02.439736 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" podUID="b9cf40c6-297f-4f91-b813-1f04685df5e0" containerName="ensemble-graph-b0164" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:33:06.089736 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:06.089709 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" Jul 28 17:33:06.178241 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:06.178198 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b9cf40c6-297f-4f91-b813-1f04685df5e0-proxy-tls\") pod \"b9cf40c6-297f-4f91-b813-1f04685df5e0\" (UID: \"b9cf40c6-297f-4f91-b813-1f04685df5e0\") " Jul 28 17:33:06.178241 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:06.178249 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9cf40c6-297f-4f91-b813-1f04685df5e0-openshift-service-ca-bundle\") pod \"b9cf40c6-297f-4f91-b813-1f04685df5e0\" (UID: \"b9cf40c6-297f-4f91-b813-1f04685df5e0\") " Jul 28 17:33:06.178672 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:06.178645 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9cf40c6-297f-4f91-b813-1f04685df5e0-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "b9cf40c6-297f-4f91-b813-1f04685df5e0" (UID: "b9cf40c6-297f-4f91-b813-1f04685df5e0"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:33:06.180323 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:06.180299 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9cf40c6-297f-4f91-b813-1f04685df5e0-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b9cf40c6-297f-4f91-b813-1f04685df5e0" (UID: "b9cf40c6-297f-4f91-b813-1f04685df5e0"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:33:06.279581 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:06.279542 2550 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b9cf40c6-297f-4f91-b813-1f04685df5e0-proxy-tls\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:33:06.279581 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:06.279579 2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b9cf40c6-297f-4f91-b813-1f04685df5e0-openshift-service-ca-bundle\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:33:06.607851 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:06.607742 2550 generic.go:356] "Generic (PLEG): container finished" podID="b9cf40c6-297f-4f91-b813-1f04685df5e0" containerID="44bfc5615778e143c51e4efe0753aa9bd88757b048f283b2d9a1cfcf58bc1f60" exitCode=0 Jul 28 17:33:06.607851 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:06.607794 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" Jul 28 17:33:06.607851 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:06.607832 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" event={"ID":"b9cf40c6-297f-4f91-b813-1f04685df5e0","Type":"ContainerDied","Data":"44bfc5615778e143c51e4efe0753aa9bd88757b048f283b2d9a1cfcf58bc1f60"} Jul 28 17:33:06.608205 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:06.607889 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7" event={"ID":"b9cf40c6-297f-4f91-b813-1f04685df5e0","Type":"ContainerDied","Data":"cdbb5c24927d1236cd8a357683876ad8dd7f0deb692db36f33efc64816998a9a"} Jul 28 17:33:06.608205 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:06.607911 2550 scope.go:117] "RemoveContainer" containerID="44bfc5615778e143c51e4efe0753aa9bd88757b048f283b2d9a1cfcf58bc1f60" Jul 28 17:33:06.617564 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:06.617544 2550 scope.go:117] "RemoveContainer" containerID="44bfc5615778e143c51e4efe0753aa9bd88757b048f283b2d9a1cfcf58bc1f60" Jul 28 17:33:06.617807 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:33:06.617786 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44bfc5615778e143c51e4efe0753aa9bd88757b048f283b2d9a1cfcf58bc1f60\": container with ID starting with 44bfc5615778e143c51e4efe0753aa9bd88757b048f283b2d9a1cfcf58bc1f60 not found: ID does not exist" containerID="44bfc5615778e143c51e4efe0753aa9bd88757b048f283b2d9a1cfcf58bc1f60" Jul 28 17:33:06.617848 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:06.617815 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44bfc5615778e143c51e4efe0753aa9bd88757b048f283b2d9a1cfcf58bc1f60"} err="failed to get container status \"44bfc5615778e143c51e4efe0753aa9bd88757b048f283b2d9a1cfcf58bc1f60\": rpc error: code = NotFound desc = could not find container \"44bfc5615778e143c51e4efe0753aa9bd88757b048f283b2d9a1cfcf58bc1f60\": container with ID starting with 44bfc5615778e143c51e4efe0753aa9bd88757b048f283b2d9a1cfcf58bc1f60 not found: ID does not exist" Jul 28 17:33:06.631431 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:06.631406 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7"] Jul 28 17:33:06.634613 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:06.634588 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-b0164-855bf76f5c-9lgt7"] Jul 28 17:33:07.771307 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:07.771269 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9cf40c6-297f-4f91-b813-1f04685df5e0" path="/var/lib/kubelet/pods/b9cf40c6-297f-4f91-b813-1f04685df5e0/volumes" Jul 28 17:33:31.646752 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:31.646714 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts"] Jul 28 17:33:31.647226 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:31.647042 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="acd97157-d6c7-49b7-b431-5b9be226c95a" containerName="sequence-graph-3a5ae" Jul 28 17:33:31.647226 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:31.647060 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="acd97157-d6c7-49b7-b431-5b9be226c95a" containerName="sequence-graph-3a5ae" Jul 28 17:33:31.647226 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:31.647085 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9cf40c6-297f-4f91-b813-1f04685df5e0" containerName="ensemble-graph-b0164" Jul 28 17:33:31.647226 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:31.647091 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9cf40c6-297f-4f91-b813-1f04685df5e0" containerName="ensemble-graph-b0164" Jul 28 17:33:31.647226 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:31.647140 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9cf40c6-297f-4f91-b813-1f04685df5e0" containerName="ensemble-graph-b0164" Jul 28 17:33:31.647226 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:31.647154 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="acd97157-d6c7-49b7-b431-5b9be226c95a" containerName="sequence-graph-3a5ae" Jul 28 17:33:31.651506 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:31.651488 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" Jul 28 17:33:31.654200 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:31.654180 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-1bf99-serving-cert\"" Jul 28 17:33:31.654315 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:31.654250 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-wxvxt\"" Jul 28 17:33:31.654533 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:31.654517 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 28 17:33:31.655498 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:31.655480 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-1bf99-kube-rbac-proxy-sar-config\"" Jul 28 17:33:31.659308 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:31.659283 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts"] Jul 28 17:33:31.690683 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:31.690645 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e66f4a8-d18c-4aa9-bd03-8511b81f2c26-openshift-service-ca-bundle\") pod \"sequence-graph-1bf99-7658f8b77-btrts\" (UID: \"9e66f4a8-d18c-4aa9-bd03-8511b81f2c26\") " pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" Jul 28 17:33:31.690862 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:31.690700 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9e66f4a8-d18c-4aa9-bd03-8511b81f2c26-proxy-tls\") pod \"sequence-graph-1bf99-7658f8b77-btrts\" (UID: \"9e66f4a8-d18c-4aa9-bd03-8511b81f2c26\") " pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" Jul 28 17:33:31.791692 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:31.791653 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e66f4a8-d18c-4aa9-bd03-8511b81f2c26-openshift-service-ca-bundle\") pod \"sequence-graph-1bf99-7658f8b77-btrts\" (UID: \"9e66f4a8-d18c-4aa9-bd03-8511b81f2c26\") " pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" Jul 28 17:33:31.791877 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:31.791716 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9e66f4a8-d18c-4aa9-bd03-8511b81f2c26-proxy-tls\") pod \"sequence-graph-1bf99-7658f8b77-btrts\" (UID: \"9e66f4a8-d18c-4aa9-bd03-8511b81f2c26\") " pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" Jul 28 17:33:31.792317 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:31.792281 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e66f4a8-d18c-4aa9-bd03-8511b81f2c26-openshift-service-ca-bundle\") pod \"sequence-graph-1bf99-7658f8b77-btrts\" (UID: \"9e66f4a8-d18c-4aa9-bd03-8511b81f2c26\") " pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" Jul 28 17:33:31.794177 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:31.794154 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9e66f4a8-d18c-4aa9-bd03-8511b81f2c26-proxy-tls\") pod \"sequence-graph-1bf99-7658f8b77-btrts\" (UID: \"9e66f4a8-d18c-4aa9-bd03-8511b81f2c26\") " pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" Jul 28 17:33:31.962138 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:31.962047 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" Jul 28 17:33:32.095840 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:32.095813 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts"] Jul 28 17:33:32.098460 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:33:32.098432 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e66f4a8_d18c_4aa9_bd03_8511b81f2c26.slice/crio-be3a5418b94c5bcbb0186c645fa8ffacda0d1da33bdedbc0bff4c378ed433426 WatchSource:0}: Error finding container be3a5418b94c5bcbb0186c645fa8ffacda0d1da33bdedbc0bff4c378ed433426: Status 404 returned error can't find the container with id be3a5418b94c5bcbb0186c645fa8ffacda0d1da33bdedbc0bff4c378ed433426 Jul 28 17:33:32.721242 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:32.721206 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" event={"ID":"9e66f4a8-d18c-4aa9-bd03-8511b81f2c26","Type":"ContainerStarted","Data":"80da4451e2854ecdc90467a929c920afcf6f41c9bb1ec6da76a0445e1dd65f27"} Jul 28 17:33:32.721748 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:32.721269 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" event={"ID":"9e66f4a8-d18c-4aa9-bd03-8511b81f2c26","Type":"ContainerStarted","Data":"be3a5418b94c5bcbb0186c645fa8ffacda0d1da33bdedbc0bff4c378ed433426"} Jul 28 17:33:32.721748 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:32.721298 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" Jul 28 17:33:32.740579 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:32.740525 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" podStartSLOduration=1.7405079730000002 podStartE2EDuration="1.740507973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:33:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:33:32.738480325 +0000 UTC m=+1397.466981685" watchObservedRunningTime="2026-07-28 17:33:32.740507973 +0000 UTC m=+1397.469009309" Jul 28 17:33:36.210168 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:36.210134 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6"] Jul 28 17:33:36.213588 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:36.213558 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" Jul 28 17:33:36.216433 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:36.216378 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-fca3b-serving-cert\"" Jul 28 17:33:36.216549 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:36.216410 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"ensemble-graph-fca3b-kube-rbac-proxy-sar-config\"" Jul 28 17:33:36.221928 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:36.221904 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6"] Jul 28 17:33:36.228519 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:36.228491 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78455cfc-f209-4387-b557-2cecb243fbad-openshift-service-ca-bundle\") pod \"ensemble-graph-fca3b-6cff5d8565-22kw6\" (UID: \"78455cfc-f209-4387-b557-2cecb243fbad\") " pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" Jul 28 17:33:36.228648 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:36.228526 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78455cfc-f209-4387-b557-2cecb243fbad-proxy-tls\") pod \"ensemble-graph-fca3b-6cff5d8565-22kw6\" (UID: \"78455cfc-f209-4387-b557-2cecb243fbad\") " pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" Jul 28 17:33:36.328922 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:36.328887 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78455cfc-f209-4387-b557-2cecb243fbad-openshift-service-ca-bundle\") pod \"ensemble-graph-fca3b-6cff5d8565-22kw6\" (UID: \"78455cfc-f209-4387-b557-2cecb243fbad\") " pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" Jul 28 17:33:36.328922 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:36.328925 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78455cfc-f209-4387-b557-2cecb243fbad-proxy-tls\") pod \"ensemble-graph-fca3b-6cff5d8565-22kw6\" (UID: \"78455cfc-f209-4387-b557-2cecb243fbad\") " pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" Jul 28 17:33:36.329148 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:33:36.329035 2550 secret.go:189] Couldn't get secret kserve-ci-e2e-test/ensemble-graph-fca3b-serving-cert: secret "ensemble-graph-fca3b-serving-cert" not found Jul 28 17:33:36.329148 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:33:36.329091 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78455cfc-f209-4387-b557-2cecb243fbad-proxy-tls podName:78455cfc-f209-4387-b557-2cecb243fbad nodeName:}" failed. No retries permitted until 2026-07-28 17:33:36.829073847 +0000 UTC m=+1401.557575160 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/78455cfc-f209-4387-b557-2cecb243fbad-proxy-tls") pod "ensemble-graph-fca3b-6cff5d8565-22kw6" (UID: "78455cfc-f209-4387-b557-2cecb243fbad") : secret "ensemble-graph-fca3b-serving-cert" not found Jul 28 17:33:36.329592 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:36.329570 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78455cfc-f209-4387-b557-2cecb243fbad-openshift-service-ca-bundle\") pod \"ensemble-graph-fca3b-6cff5d8565-22kw6\" (UID: \"78455cfc-f209-4387-b557-2cecb243fbad\") " pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" Jul 28 17:33:36.833629 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:36.833589 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78455cfc-f209-4387-b557-2cecb243fbad-proxy-tls\") pod \"ensemble-graph-fca3b-6cff5d8565-22kw6\" (UID: \"78455cfc-f209-4387-b557-2cecb243fbad\") " pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" Jul 28 17:33:36.835996 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:36.835970 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78455cfc-f209-4387-b557-2cecb243fbad-proxy-tls\") pod \"ensemble-graph-fca3b-6cff5d8565-22kw6\" (UID: \"78455cfc-f209-4387-b557-2cecb243fbad\") " pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" Jul 28 17:33:37.124340 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:37.124255 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" Jul 28 17:33:37.251608 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:37.251583 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6"] Jul 28 17:33:37.253660 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:33:37.253634 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78455cfc_f209_4387_b557_2cecb243fbad.slice/crio-fad01803d5adcb49121c86a86b9d4c2f60fdfc63c62e9decc4cf5d7abe90e5d0 WatchSource:0}: Error finding container fad01803d5adcb49121c86a86b9d4c2f60fdfc63c62e9decc4cf5d7abe90e5d0: Status 404 returned error can't find the container with id fad01803d5adcb49121c86a86b9d4c2f60fdfc63c62e9decc4cf5d7abe90e5d0 Jul 28 17:33:37.746319 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:37.746283 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" event={"ID":"78455cfc-f209-4387-b557-2cecb243fbad","Type":"ContainerStarted","Data":"25063633d12d579de0d2936cd8cbd65b409a7aa5e67bd452ecf0a5cbdc7f8c1d"} Jul 28 17:33:37.746514 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:37.746336 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" event={"ID":"78455cfc-f209-4387-b557-2cecb243fbad","Type":"ContainerStarted","Data":"fad01803d5adcb49121c86a86b9d4c2f60fdfc63c62e9decc4cf5d7abe90e5d0"} Jul 28 17:33:37.746514 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:37.746504 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" Jul 28 17:33:37.767540 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:37.767486 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" podStartSLOduration=1.7674687329999998 podStartE2EDuration="1.767468733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:33:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:33:37.76528325 +0000 UTC m=+1402.493784586" watchObservedRunningTime="2026-07-28 17:33:37.767468733 +0000 UTC m=+1402.495970070" Jul 28 17:33:38.732254 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:38.732218 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" Jul 28 17:33:43.757608 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:43.757580 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" Jul 28 17:33:45.705976 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:45.705943 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts"] Jul 28 17:33:45.706488 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:45.706172 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" podUID="9e66f4a8-d18c-4aa9-bd03-8511b81f2c26" containerName="sequence-graph-1bf99" containerID="cri-o://80da4451e2854ecdc90467a929c920afcf6f41c9bb1ec6da76a0445e1dd65f27" gracePeriod=30 Jul 28 17:33:48.731115 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:48.731077 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" podUID="9e66f4a8-d18c-4aa9-bd03-8511b81f2c26" containerName="sequence-graph-1bf99" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:33:53.731138 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:53.731087 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" podUID="9e66f4a8-d18c-4aa9-bd03-8511b81f2c26" containerName="sequence-graph-1bf99" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:33:58.730151 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:58.730102 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" podUID="9e66f4a8-d18c-4aa9-bd03-8511b81f2c26" containerName="sequence-graph-1bf99" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:33:58.730649 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:33:58.730215 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" Jul 28 17:34:03.730595 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:03.730553 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" podUID="9e66f4a8-d18c-4aa9-bd03-8511b81f2c26" containerName="sequence-graph-1bf99" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:34:08.730937 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:08.730848 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" podUID="9e66f4a8-d18c-4aa9-bd03-8511b81f2c26" containerName="sequence-graph-1bf99" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:34:13.730826 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:13.730783 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" podUID="9e66f4a8-d18c-4aa9-bd03-8511b81f2c26" containerName="sequence-graph-1bf99" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:34:15.853847 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:15.853814 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" Jul 28 17:34:15.912413 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:15.912357 2550 generic.go:356] "Generic (PLEG): container finished" podID="9e66f4a8-d18c-4aa9-bd03-8511b81f2c26" containerID="80da4451e2854ecdc90467a929c920afcf6f41c9bb1ec6da76a0445e1dd65f27" exitCode=0 Jul 28 17:34:15.912595 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:15.912421 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" Jul 28 17:34:15.912595 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:15.912447 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" event={"ID":"9e66f4a8-d18c-4aa9-bd03-8511b81f2c26","Type":"ContainerDied","Data":"80da4451e2854ecdc90467a929c920afcf6f41c9bb1ec6da76a0445e1dd65f27"} Jul 28 17:34:15.912595 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:15.912503 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts" event={"ID":"9e66f4a8-d18c-4aa9-bd03-8511b81f2c26","Type":"ContainerDied","Data":"be3a5418b94c5bcbb0186c645fa8ffacda0d1da33bdedbc0bff4c378ed433426"} Jul 28 17:34:15.912595 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:15.912526 2550 scope.go:117] "RemoveContainer" containerID="80da4451e2854ecdc90467a929c920afcf6f41c9bb1ec6da76a0445e1dd65f27" Jul 28 17:34:15.922816 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:15.922794 2550 scope.go:117] "RemoveContainer" containerID="80da4451e2854ecdc90467a929c920afcf6f41c9bb1ec6da76a0445e1dd65f27" Jul 28 17:34:15.923141 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:34:15.923118 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80da4451e2854ecdc90467a929c920afcf6f41c9bb1ec6da76a0445e1dd65f27\": container with ID starting with 80da4451e2854ecdc90467a929c920afcf6f41c9bb1ec6da76a0445e1dd65f27 not found: ID does not exist" containerID="80da4451e2854ecdc90467a929c920afcf6f41c9bb1ec6da76a0445e1dd65f27" Jul 28 17:34:15.923220 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:15.923155 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80da4451e2854ecdc90467a929c920afcf6f41c9bb1ec6da76a0445e1dd65f27"} err="failed to get container status \"80da4451e2854ecdc90467a929c920afcf6f41c9bb1ec6da76a0445e1dd65f27\": rpc error: code = NotFound desc = could not find container \"80da4451e2854ecdc90467a929c920afcf6f41c9bb1ec6da76a0445e1dd65f27\": container with ID starting with 80da4451e2854ecdc90467a929c920afcf6f41c9bb1ec6da76a0445e1dd65f27 not found: ID does not exist" Jul 28 17:34:15.963579 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:15.963489 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9e66f4a8-d18c-4aa9-bd03-8511b81f2c26-proxy-tls\") pod \"9e66f4a8-d18c-4aa9-bd03-8511b81f2c26\" (UID: \"9e66f4a8-d18c-4aa9-bd03-8511b81f2c26\") " Jul 28 17:34:15.963579 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:15.963528 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e66f4a8-d18c-4aa9-bd03-8511b81f2c26-openshift-service-ca-bundle\") pod \"9e66f4a8-d18c-4aa9-bd03-8511b81f2c26\" (UID: \"9e66f4a8-d18c-4aa9-bd03-8511b81f2c26\") " Jul 28 17:34:15.963909 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:15.963884 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e66f4a8-d18c-4aa9-bd03-8511b81f2c26-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "9e66f4a8-d18c-4aa9-bd03-8511b81f2c26" (UID: "9e66f4a8-d18c-4aa9-bd03-8511b81f2c26"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:34:15.965613 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:15.965591 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e66f4a8-d18c-4aa9-bd03-8511b81f2c26-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "9e66f4a8-d18c-4aa9-bd03-8511b81f2c26" (UID: "9e66f4a8-d18c-4aa9-bd03-8511b81f2c26"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:34:16.065160 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:16.065122 2550 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/9e66f4a8-d18c-4aa9-bd03-8511b81f2c26-proxy-tls\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:34:16.065160 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:16.065156 2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9e66f4a8-d18c-4aa9-bd03-8511b81f2c26-openshift-service-ca-bundle\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:34:16.236099 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:16.236060 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts"] Jul 28 17:34:16.240239 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:16.240208 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-1bf99-7658f8b77-btrts"] Jul 28 17:34:17.771856 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:17.771818 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e66f4a8-d18c-4aa9-bd03-8511b81f2c26" path="/var/lib/kubelet/pods/9e66f4a8-d18c-4aa9-bd03-8511b81f2c26/volumes" Jul 28 17:34:45.966716 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:45.966678 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp"] Jul 28 17:34:45.967168 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:45.966987 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e66f4a8-d18c-4aa9-bd03-8511b81f2c26" containerName="sequence-graph-1bf99" Jul 28 17:34:45.967168 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:45.966998 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e66f4a8-d18c-4aa9-bd03-8511b81f2c26" containerName="sequence-graph-1bf99" Jul 28 17:34:45.967168 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:45.967062 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e66f4a8-d18c-4aa9-bd03-8511b81f2c26" containerName="sequence-graph-1bf99" Jul 28 17:34:45.969926 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:45.969908 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" Jul 28 17:34:45.972695 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:45.972671 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-9d7bb-kube-rbac-proxy-sar-config\"" Jul 28 17:34:45.972695 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:45.972692 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"sequence-graph-9d7bb-serving-cert\"" Jul 28 17:34:45.978221 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:45.978197 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp"] Jul 28 17:34:45.984127 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:45.984099 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/39de043e-47ed-4211-93c5-1c76105cd383-proxy-tls\") pod \"sequence-graph-9d7bb-645c799655-ks4zp\" (UID: \"39de043e-47ed-4211-93c5-1c76105cd383\") " pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" Jul 28 17:34:45.984229 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:45.984177 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39de043e-47ed-4211-93c5-1c76105cd383-openshift-service-ca-bundle\") pod \"sequence-graph-9d7bb-645c799655-ks4zp\" (UID: \"39de043e-47ed-4211-93c5-1c76105cd383\") " pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" Jul 28 17:34:46.085442 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:46.085375 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39de043e-47ed-4211-93c5-1c76105cd383-openshift-service-ca-bundle\") pod \"sequence-graph-9d7bb-645c799655-ks4zp\" (UID: \"39de043e-47ed-4211-93c5-1c76105cd383\") " pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" Jul 28 17:34:46.085642 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:46.085496 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/39de043e-47ed-4211-93c5-1c76105cd383-proxy-tls\") pod \"sequence-graph-9d7bb-645c799655-ks4zp\" (UID: \"39de043e-47ed-4211-93c5-1c76105cd383\") " pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" Jul 28 17:34:46.085642 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:34:46.085609 2550 secret.go:189] Couldn't get secret kserve-ci-e2e-test/sequence-graph-9d7bb-serving-cert: secret "sequence-graph-9d7bb-serving-cert" not found Jul 28 17:34:46.085760 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:34:46.085680 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39de043e-47ed-4211-93c5-1c76105cd383-proxy-tls podName:39de043e-47ed-4211-93c5-1c76105cd383 nodeName:}" failed. No retries permitted until 2026-07-28 17:34:46.585658703 +0000 UTC m=+1471.314160032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/39de043e-47ed-4211-93c5-1c76105cd383-proxy-tls") pod "sequence-graph-9d7bb-645c799655-ks4zp" (UID: "39de043e-47ed-4211-93c5-1c76105cd383") : secret "sequence-graph-9d7bb-serving-cert" not found Jul 28 17:34:46.086064 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:46.086042 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39de043e-47ed-4211-93c5-1c76105cd383-openshift-service-ca-bundle\") pod \"sequence-graph-9d7bb-645c799655-ks4zp\" (UID: \"39de043e-47ed-4211-93c5-1c76105cd383\") " pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" Jul 28 17:34:46.588336 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:46.588294 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/39de043e-47ed-4211-93c5-1c76105cd383-proxy-tls\") pod \"sequence-graph-9d7bb-645c799655-ks4zp\" (UID: \"39de043e-47ed-4211-93c5-1c76105cd383\") " pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" Jul 28 17:34:46.590842 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:46.590815 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/39de043e-47ed-4211-93c5-1c76105cd383-proxy-tls\") pod \"sequence-graph-9d7bb-645c799655-ks4zp\" (UID: \"39de043e-47ed-4211-93c5-1c76105cd383\") " pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" Jul 28 17:34:46.880037 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:46.879948 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" Jul 28 17:34:47.008514 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:47.008480 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp"] Jul 28 17:34:47.011572 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:34:47.011540 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39de043e_47ed_4211_93c5_1c76105cd383.slice/crio-2d6dfb2632a966393a21ae5e792bed5031578b95b328b516ea203bc795638bad WatchSource:0}: Error finding container 2d6dfb2632a966393a21ae5e792bed5031578b95b328b516ea203bc795638bad: Status 404 returned error can't find the container with id 2d6dfb2632a966393a21ae5e792bed5031578b95b328b516ea203bc795638bad Jul 28 17:34:47.051445 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:47.051414 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" event={"ID":"39de043e-47ed-4211-93c5-1c76105cd383","Type":"ContainerStarted","Data":"2d6dfb2632a966393a21ae5e792bed5031578b95b328b516ea203bc795638bad"} Jul 28 17:34:48.056787 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:48.056745 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" event={"ID":"39de043e-47ed-4211-93c5-1c76105cd383","Type":"ContainerStarted","Data":"45edab0452201b3e61f78654b7c109b327148f0d74dd94fb4983c5979c2701ef"} Jul 28 17:34:48.057185 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:48.056864 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" Jul 28 17:34:48.074810 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:48.074729 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" podStartSLOduration=3.074711127 podStartE2EDuration="3.074711127s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:34:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:34:48.07351787 +0000 UTC m=+1472.802019205" watchObservedRunningTime="2026-07-28 17:34:48.074711127 +0000 UTC m=+1472.803212513" Jul 28 17:34:54.066601 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:34:54.066569 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" Jul 28 17:41:50.898266 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:41:50.898228 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6"] Jul 28 17:41:50.900763 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:41:50.898476 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" podUID="78455cfc-f209-4387-b557-2cecb243fbad" containerName="ensemble-graph-fca3b" containerID="cri-o://25063633d12d579de0d2936cd8cbd65b409a7aa5e67bd452ecf0a5cbdc7f8c1d" gracePeriod=30 Jul 28 17:41:53.755071 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:41:53.755034 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" podUID="78455cfc-f209-4387-b557-2cecb243fbad" containerName="ensemble-graph-fca3b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:41:58.755446 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:41:58.755375 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" podUID="78455cfc-f209-4387-b557-2cecb243fbad" containerName="ensemble-graph-fca3b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:42:03.755142 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:03.755094 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" podUID="78455cfc-f209-4387-b557-2cecb243fbad" containerName="ensemble-graph-fca3b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:42:03.755568 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:03.755200 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" Jul 28 17:42:08.755770 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:08.755720 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" podUID="78455cfc-f209-4387-b557-2cecb243fbad" containerName="ensemble-graph-fca3b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:42:13.755203 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:13.755162 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" podUID="78455cfc-f209-4387-b557-2cecb243fbad" containerName="ensemble-graph-fca3b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:42:18.755354 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:18.755313 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" podUID="78455cfc-f209-4387-b557-2cecb243fbad" containerName="ensemble-graph-fca3b" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:42:20.982547 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:20.982511 2550 generic.go:356] "Generic (PLEG): container finished" podID="78455cfc-f209-4387-b557-2cecb243fbad" containerID="25063633d12d579de0d2936cd8cbd65b409a7aa5e67bd452ecf0a5cbdc7f8c1d" exitCode=0 Jul 28 17:42:20.982889 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:20.982573 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" event={"ID":"78455cfc-f209-4387-b557-2cecb243fbad","Type":"ContainerDied","Data":"25063633d12d579de0d2936cd8cbd65b409a7aa5e67bd452ecf0a5cbdc7f8c1d"} Jul 28 17:42:21.054698 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:21.054653 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" Jul 28 17:42:21.141000 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:21.140967 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78455cfc-f209-4387-b557-2cecb243fbad-proxy-tls\") pod \"78455cfc-f209-4387-b557-2cecb243fbad\" (UID: \"78455cfc-f209-4387-b557-2cecb243fbad\") " Jul 28 17:42:21.141178 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:21.141042 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78455cfc-f209-4387-b557-2cecb243fbad-openshift-service-ca-bundle\") pod \"78455cfc-f209-4387-b557-2cecb243fbad\" (UID: \"78455cfc-f209-4387-b557-2cecb243fbad\") " Jul 28 17:42:21.141353 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:21.141329 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78455cfc-f209-4387-b557-2cecb243fbad-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "78455cfc-f209-4387-b557-2cecb243fbad" (UID: "78455cfc-f209-4387-b557-2cecb243fbad"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:42:21.143099 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:21.143080 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78455cfc-f209-4387-b557-2cecb243fbad-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "78455cfc-f209-4387-b557-2cecb243fbad" (UID: "78455cfc-f209-4387-b557-2cecb243fbad"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:42:21.241699 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:21.241613 2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78455cfc-f209-4387-b557-2cecb243fbad-openshift-service-ca-bundle\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:42:21.241699 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:21.241646 2550 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/78455cfc-f209-4387-b557-2cecb243fbad-proxy-tls\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:42:21.988758 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:21.988727 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" Jul 28 17:42:21.988758 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:21.988736 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6" event={"ID":"78455cfc-f209-4387-b557-2cecb243fbad","Type":"ContainerDied","Data":"fad01803d5adcb49121c86a86b9d4c2f60fdfc63c62e9decc4cf5d7abe90e5d0"} Jul 28 17:42:21.989267 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:21.988792 2550 scope.go:117] "RemoveContainer" containerID="25063633d12d579de0d2936cd8cbd65b409a7aa5e67bd452ecf0a5cbdc7f8c1d" Jul 28 17:42:22.007882 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:22.007855 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6"] Jul 28 17:42:22.012033 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:22.012008 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/ensemble-graph-fca3b-6cff5d8565-22kw6"] Jul 28 17:42:23.772678 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:23.772641 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78455cfc-f209-4387-b557-2cecb243fbad" path="/var/lib/kubelet/pods/78455cfc-f209-4387-b557-2cecb243fbad/volumes" Jul 28 17:42:51.181203 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:51.181166 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh"] Jul 28 17:42:51.181709 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:51.181527 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78455cfc-f209-4387-b557-2cecb243fbad" containerName="ensemble-graph-fca3b" Jul 28 17:42:51.181709 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:51.181539 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="78455cfc-f209-4387-b557-2cecb243fbad" containerName="ensemble-graph-fca3b" Jul 28 17:42:51.181709 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:51.181593 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="78455cfc-f209-4387-b557-2cecb243fbad" containerName="ensemble-graph-fca3b" Jul 28 17:42:51.184351 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:51.184334 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" Jul 28 17:42:51.187561 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:51.187538 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-b5d39-kube-rbac-proxy-sar-config\"" Jul 28 17:42:51.187679 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:51.187545 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-b5d39-serving-cert\"" Jul 28 17:42:51.193224 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:51.193201 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh"] Jul 28 17:42:51.284005 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:51.283966 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ced0663-42d5-480d-ba09-ced7de0f4d58-proxy-tls\") pod \"splitter-graph-b5d39-69cfbcb448-srpzh\" (UID: \"5ced0663-42d5-480d-ba09-ced7de0f4d58\") " pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" Jul 28 17:42:51.284180 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:51.284019 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ced0663-42d5-480d-ba09-ced7de0f4d58-openshift-service-ca-bundle\") pod \"splitter-graph-b5d39-69cfbcb448-srpzh\" (UID: \"5ced0663-42d5-480d-ba09-ced7de0f4d58\") " pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" Jul 28 17:42:51.384489 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:51.384456 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ced0663-42d5-480d-ba09-ced7de0f4d58-openshift-service-ca-bundle\") pod \"splitter-graph-b5d39-69cfbcb448-srpzh\" (UID: \"5ced0663-42d5-480d-ba09-ced7de0f4d58\") " pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" Jul 28 17:42:51.384697 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:51.384557 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ced0663-42d5-480d-ba09-ced7de0f4d58-proxy-tls\") pod \"splitter-graph-b5d39-69cfbcb448-srpzh\" (UID: \"5ced0663-42d5-480d-ba09-ced7de0f4d58\") " pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" Jul 28 17:42:51.385176 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:51.385149 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ced0663-42d5-480d-ba09-ced7de0f4d58-openshift-service-ca-bundle\") pod \"splitter-graph-b5d39-69cfbcb448-srpzh\" (UID: \"5ced0663-42d5-480d-ba09-ced7de0f4d58\") " pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" Jul 28 17:42:51.387108 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:51.387071 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ced0663-42d5-480d-ba09-ced7de0f4d58-proxy-tls\") pod \"splitter-graph-b5d39-69cfbcb448-srpzh\" (UID: \"5ced0663-42d5-480d-ba09-ced7de0f4d58\") " pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" Jul 28 17:42:51.495161 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:51.495055 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" Jul 28 17:42:51.630042 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:51.630016 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh"] Jul 28 17:42:51.632557 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:42:51.632529 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ced0663_42d5_480d_ba09_ced7de0f4d58.slice/crio-4f678a90be50f1886fbf68954b100653487fa916b1c39fbecd45d0294c7a868b WatchSource:0}: Error finding container 4f678a90be50f1886fbf68954b100653487fa916b1c39fbecd45d0294c7a868b: Status 404 returned error can't find the container with id 4f678a90be50f1886fbf68954b100653487fa916b1c39fbecd45d0294c7a868b Jul 28 17:42:52.116342 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:52.116306 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" event={"ID":"5ced0663-42d5-480d-ba09-ced7de0f4d58","Type":"ContainerStarted","Data":"28668579bf134dd454de484928170f90a14d2bb5baedf24fab97ea93c9f4df11"} Jul 28 17:42:52.116541 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:52.116362 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" event={"ID":"5ced0663-42d5-480d-ba09-ced7de0f4d58","Type":"ContainerStarted","Data":"4f678a90be50f1886fbf68954b100653487fa916b1c39fbecd45d0294c7a868b"} Jul 28 17:42:52.116541 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:52.116431 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" Jul 28 17:42:52.137728 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:52.137672 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" podStartSLOduration=1.1376530360000001 podStartE2EDuration="1.137653036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:42:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:42:52.137353556 +0000 UTC m=+1956.865854892" watchObservedRunningTime="2026-07-28 17:42:52.137653036 +0000 UTC m=+1956.866154371" Jul 28 17:42:58.126511 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:42:58.126482 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" Jul 28 17:43:00.660104 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:00.660067 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp"] Jul 28 17:43:00.660979 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:00.660942 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" podUID="39de043e-47ed-4211-93c5-1c76105cd383" containerName="sequence-graph-9d7bb" containerID="cri-o://45edab0452201b3e61f78654b7c109b327148f0d74dd94fb4983c5979c2701ef" gracePeriod=30 Jul 28 17:43:01.245266 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:01.245231 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh"] Jul 28 17:43:01.245482 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:01.245456 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" podUID="5ced0663-42d5-480d-ba09-ced7de0f4d58" containerName="splitter-graph-b5d39" containerID="cri-o://28668579bf134dd454de484928170f90a14d2bb5baedf24fab97ea93c9f4df11" gracePeriod=30 Jul 28 17:43:03.125451 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:03.125408 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" podUID="5ced0663-42d5-480d-ba09-ced7de0f4d58" containerName="splitter-graph-b5d39" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:43:04.065230 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:04.065188 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" podUID="39de043e-47ed-4211-93c5-1c76105cd383" containerName="sequence-graph-9d7bb" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:43:08.124759 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:08.124672 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" podUID="5ced0663-42d5-480d-ba09-ced7de0f4d58" containerName="splitter-graph-b5d39" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:43:09.065798 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:09.065759 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" podUID="39de043e-47ed-4211-93c5-1c76105cd383" containerName="sequence-graph-9d7bb" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:43:13.124986 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:13.124943 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" podUID="5ced0663-42d5-480d-ba09-ced7de0f4d58" containerName="splitter-graph-b5d39" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:43:13.125382 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:13.125055 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" Jul 28 17:43:14.065665 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:14.065626 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" podUID="39de043e-47ed-4211-93c5-1c76105cd383" containerName="sequence-graph-9d7bb" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:43:14.065871 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:14.065734 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" Jul 28 17:43:18.124624 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:18.124581 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" podUID="5ced0663-42d5-480d-ba09-ced7de0f4d58" containerName="splitter-graph-b5d39" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:43:19.065509 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:19.065465 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" podUID="39de043e-47ed-4211-93c5-1c76105cd383" containerName="sequence-graph-9d7bb" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:43:23.125496 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:23.125457 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" podUID="5ced0663-42d5-480d-ba09-ced7de0f4d58" containerName="splitter-graph-b5d39" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:43:24.064815 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:24.064774 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" podUID="39de043e-47ed-4211-93c5-1c76105cd383" containerName="sequence-graph-9d7bb" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:43:28.124837 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:28.124797 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" podUID="5ced0663-42d5-480d-ba09-ced7de0f4d58" containerName="splitter-graph-b5d39" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:43:29.065757 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:29.065718 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" podUID="39de043e-47ed-4211-93c5-1c76105cd383" containerName="sequence-graph-9d7bb" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:43:30.813349 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:30.813316 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" Jul 28 17:43:30.884556 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:30.884523 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39de043e-47ed-4211-93c5-1c76105cd383-openshift-service-ca-bundle\") pod \"39de043e-47ed-4211-93c5-1c76105cd383\" (UID: \"39de043e-47ed-4211-93c5-1c76105cd383\") " Jul 28 17:43:30.884765 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:30.884590 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/39de043e-47ed-4211-93c5-1c76105cd383-proxy-tls\") pod \"39de043e-47ed-4211-93c5-1c76105cd383\" (UID: \"39de043e-47ed-4211-93c5-1c76105cd383\") " Jul 28 17:43:30.884964 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:30.884943 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39de043e-47ed-4211-93c5-1c76105cd383-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "39de043e-47ed-4211-93c5-1c76105cd383" (UID: "39de043e-47ed-4211-93c5-1c76105cd383"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:43:30.886805 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:30.886778 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39de043e-47ed-4211-93c5-1c76105cd383-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "39de043e-47ed-4211-93c5-1c76105cd383" (UID: "39de043e-47ed-4211-93c5-1c76105cd383"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:43:30.985249 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:30.985152 2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39de043e-47ed-4211-93c5-1c76105cd383-openshift-service-ca-bundle\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:43:30.985249 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:30.985191 2550 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/39de043e-47ed-4211-93c5-1c76105cd383-proxy-tls\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:43:31.288688 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:31.288534 2550 generic.go:356] "Generic (PLEG): container finished" podID="39de043e-47ed-4211-93c5-1c76105cd383" containerID="45edab0452201b3e61f78654b7c109b327148f0d74dd94fb4983c5979c2701ef" exitCode=0 Jul 28 17:43:31.288688 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:31.288586 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" Jul 28 17:43:31.288688 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:31.288586 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" event={"ID":"39de043e-47ed-4211-93c5-1c76105cd383","Type":"ContainerDied","Data":"45edab0452201b3e61f78654b7c109b327148f0d74dd94fb4983c5979c2701ef"} Jul 28 17:43:31.288688 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:31.288656 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp" event={"ID":"39de043e-47ed-4211-93c5-1c76105cd383","Type":"ContainerDied","Data":"2d6dfb2632a966393a21ae5e792bed5031578b95b328b516ea203bc795638bad"} Jul 28 17:43:31.288688 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:31.288688 2550 scope.go:117] "RemoveContainer" containerID="45edab0452201b3e61f78654b7c109b327148f0d74dd94fb4983c5979c2701ef" Jul 28 17:43:31.297457 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:31.297424 2550 generic.go:356] "Generic (PLEG): container finished" podID="5ced0663-42d5-480d-ba09-ced7de0f4d58" containerID="28668579bf134dd454de484928170f90a14d2bb5baedf24fab97ea93c9f4df11" exitCode=0 Jul 28 17:43:31.297614 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:31.297585 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" event={"ID":"5ced0663-42d5-480d-ba09-ced7de0f4d58","Type":"ContainerDied","Data":"28668579bf134dd454de484928170f90a14d2bb5baedf24fab97ea93c9f4df11"} Jul 28 17:43:31.304671 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:31.304643 2550 scope.go:117] "RemoveContainer" containerID="45edab0452201b3e61f78654b7c109b327148f0d74dd94fb4983c5979c2701ef" Jul 28 17:43:31.304997 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:43:31.304976 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45edab0452201b3e61f78654b7c109b327148f0d74dd94fb4983c5979c2701ef\": container with ID starting with 45edab0452201b3e61f78654b7c109b327148f0d74dd94fb4983c5979c2701ef not found: ID does not exist" containerID="45edab0452201b3e61f78654b7c109b327148f0d74dd94fb4983c5979c2701ef" Jul 28 17:43:31.305069 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:31.305004 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45edab0452201b3e61f78654b7c109b327148f0d74dd94fb4983c5979c2701ef"} err="failed to get container status \"45edab0452201b3e61f78654b7c109b327148f0d74dd94fb4983c5979c2701ef\": rpc error: code = NotFound desc = could not find container \"45edab0452201b3e61f78654b7c109b327148f0d74dd94fb4983c5979c2701ef\": container with ID starting with 45edab0452201b3e61f78654b7c109b327148f0d74dd94fb4983c5979c2701ef not found: ID does not exist" Jul 28 17:43:31.316500 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:31.316476 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp"] Jul 28 17:43:31.322493 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:31.322472 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/sequence-graph-9d7bb-645c799655-ks4zp"] Jul 28 17:43:31.385955 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:31.385928 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" Jul 28 17:43:31.487894 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:31.487851 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ced0663-42d5-480d-ba09-ced7de0f4d58-proxy-tls\") pod \"5ced0663-42d5-480d-ba09-ced7de0f4d58\" (UID: \"5ced0663-42d5-480d-ba09-ced7de0f4d58\") " Jul 28 17:43:31.488070 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:31.487910 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ced0663-42d5-480d-ba09-ced7de0f4d58-openshift-service-ca-bundle\") pod \"5ced0663-42d5-480d-ba09-ced7de0f4d58\" (UID: \"5ced0663-42d5-480d-ba09-ced7de0f4d58\") " Jul 28 17:43:31.488270 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:31.488243 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ced0663-42d5-480d-ba09-ced7de0f4d58-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "5ced0663-42d5-480d-ba09-ced7de0f4d58" (UID: "5ced0663-42d5-480d-ba09-ced7de0f4d58"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:43:31.489984 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:31.489966 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ced0663-42d5-480d-ba09-ced7de0f4d58-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "5ced0663-42d5-480d-ba09-ced7de0f4d58" (UID: "5ced0663-42d5-480d-ba09-ced7de0f4d58"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:43:31.589244 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:31.589206 2550 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/5ced0663-42d5-480d-ba09-ced7de0f4d58-proxy-tls\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:43:31.589244 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:31.589237 2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ced0663-42d5-480d-ba09-ced7de0f4d58-openshift-service-ca-bundle\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:43:31.771790 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:31.771756 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39de043e-47ed-4211-93c5-1c76105cd383" path="/var/lib/kubelet/pods/39de043e-47ed-4211-93c5-1c76105cd383/volumes" Jul 28 17:43:32.303128 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:32.303088 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" event={"ID":"5ced0663-42d5-480d-ba09-ced7de0f4d58","Type":"ContainerDied","Data":"4f678a90be50f1886fbf68954b100653487fa916b1c39fbecd45d0294c7a868b"} Jul 28 17:43:32.303634 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:32.303104 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh" Jul 28 17:43:32.303634 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:32.303157 2550 scope.go:117] "RemoveContainer" containerID="28668579bf134dd454de484928170f90a14d2bb5baedf24fab97ea93c9f4df11" Jul 28 17:43:32.321227 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:32.321178 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh"] Jul 28 17:43:32.325561 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:32.325536 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-b5d39-69cfbcb448-srpzh"] Jul 28 17:43:33.771230 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:43:33.771196 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ced0663-42d5-480d-ba09-ced7de0f4d58" path="/var/lib/kubelet/pods/5ced0663-42d5-480d-ba09-ced7de0f4d58/volumes" Jul 28 17:44:01.516251 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:01.516214 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv"] Jul 28 17:44:01.516808 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:01.516672 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ced0663-42d5-480d-ba09-ced7de0f4d58" containerName="splitter-graph-b5d39" Jul 28 17:44:01.516808 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:01.516691 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ced0663-42d5-480d-ba09-ced7de0f4d58" containerName="splitter-graph-b5d39" Jul 28 17:44:01.516808 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:01.516727 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39de043e-47ed-4211-93c5-1c76105cd383" containerName="sequence-graph-9d7bb" Jul 28 17:44:01.516808 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:01.516736 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="39de043e-47ed-4211-93c5-1c76105cd383" containerName="sequence-graph-9d7bb" Jul 28 17:44:01.517019 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:01.516817 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ced0663-42d5-480d-ba09-ced7de0f4d58" containerName="splitter-graph-b5d39" Jul 28 17:44:01.517019 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:01.516834 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="39de043e-47ed-4211-93c5-1c76105cd383" containerName="sequence-graph-9d7bb" Jul 28 17:44:01.519806 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:01.519787 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" Jul 28 17:44:01.522678 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:01.522655 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 28 17:44:01.522806 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:01.522698 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-4f50a-serving-cert\"" Jul 28 17:44:01.522806 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:01.522772 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"splitter-graph-4f50a-kube-rbac-proxy-sar-config\"" Jul 28 17:44:01.522806 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:01.522778 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-wxvxt\"" Jul 28 17:44:01.527990 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:01.527968 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv"] Jul 28 17:44:01.646907 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:01.646865 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11ad9f23-e3f4-4d26-b401-37dab78becc3-openshift-service-ca-bundle\") pod \"splitter-graph-4f50a-747967f6f4-dwptv\" (UID: \"11ad9f23-e3f4-4d26-b401-37dab78becc3\") " pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" Jul 28 17:44:01.646907 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:01.646906 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/11ad9f23-e3f4-4d26-b401-37dab78becc3-proxy-tls\") pod \"splitter-graph-4f50a-747967f6f4-dwptv\" (UID: \"11ad9f23-e3f4-4d26-b401-37dab78becc3\") " pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" Jul 28 17:44:01.748307 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:01.748271 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11ad9f23-e3f4-4d26-b401-37dab78becc3-openshift-service-ca-bundle\") pod \"splitter-graph-4f50a-747967f6f4-dwptv\" (UID: \"11ad9f23-e3f4-4d26-b401-37dab78becc3\") " pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" Jul 28 17:44:01.748307 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:01.748306 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/11ad9f23-e3f4-4d26-b401-37dab78becc3-proxy-tls\") pod \"splitter-graph-4f50a-747967f6f4-dwptv\" (UID: \"11ad9f23-e3f4-4d26-b401-37dab78becc3\") " pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" Jul 28 17:44:01.748554 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:44:01.748453 2550 secret.go:189] Couldn't get secret kserve-ci-e2e-test/splitter-graph-4f50a-serving-cert: secret "splitter-graph-4f50a-serving-cert" not found Jul 28 17:44:01.748554 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:44:01.748520 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11ad9f23-e3f4-4d26-b401-37dab78becc3-proxy-tls podName:11ad9f23-e3f4-4d26-b401-37dab78becc3 nodeName:}" failed. No retries permitted until 2026-07-28 17:44:02.248501871 +0000 UTC m=+2026.977003184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/11ad9f23-e3f4-4d26-b401-37dab78becc3-proxy-tls") pod "splitter-graph-4f50a-747967f6f4-dwptv" (UID: "11ad9f23-e3f4-4d26-b401-37dab78becc3") : secret "splitter-graph-4f50a-serving-cert" not found Jul 28 17:44:01.748910 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:01.748891 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11ad9f23-e3f4-4d26-b401-37dab78becc3-openshift-service-ca-bundle\") pod \"splitter-graph-4f50a-747967f6f4-dwptv\" (UID: \"11ad9f23-e3f4-4d26-b401-37dab78becc3\") " pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" Jul 28 17:44:02.252169 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:02.252117 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/11ad9f23-e3f4-4d26-b401-37dab78becc3-proxy-tls\") pod \"splitter-graph-4f50a-747967f6f4-dwptv\" (UID: \"11ad9f23-e3f4-4d26-b401-37dab78becc3\") " pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" Jul 28 17:44:02.254708 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:02.254678 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/11ad9f23-e3f4-4d26-b401-37dab78becc3-proxy-tls\") pod \"splitter-graph-4f50a-747967f6f4-dwptv\" (UID: \"11ad9f23-e3f4-4d26-b401-37dab78becc3\") " pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" Jul 28 17:44:02.430118 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:02.430081 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" Jul 28 17:44:02.566960 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:02.566872 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv"] Jul 28 17:44:02.569251 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:44:02.569221 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11ad9f23_e3f4_4d26_b401_37dab78becc3.slice/crio-04fce6543d8814faf3cd876a0a98b892feb161e92ad0886e757a1d64505be706 WatchSource:0}: Error finding container 04fce6543d8814faf3cd876a0a98b892feb161e92ad0886e757a1d64505be706: Status 404 returned error can't find the container with id 04fce6543d8814faf3cd876a0a98b892feb161e92ad0886e757a1d64505be706 Jul 28 17:44:03.435967 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:03.435931 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" event={"ID":"11ad9f23-e3f4-4d26-b401-37dab78becc3","Type":"ContainerStarted","Data":"f665fa69e7ee1502b6b87fd192067cb48275cfc489365a3f3bf3f0914ed9aeea"} Jul 28 17:44:03.436162 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:03.435985 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" event={"ID":"11ad9f23-e3f4-4d26-b401-37dab78becc3","Type":"ContainerStarted","Data":"04fce6543d8814faf3cd876a0a98b892feb161e92ad0886e757a1d64505be706"} Jul 28 17:44:03.436162 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:03.436060 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" Jul 28 17:44:03.456273 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:03.456208 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" podStartSLOduration=2.456190908 podStartE2EDuration="2.456190908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:44:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:44:03.454871155 +0000 UTC m=+2028.183372491" watchObservedRunningTime="2026-07-28 17:44:03.456190908 +0000 UTC m=+2028.184692244" Jul 28 17:44:09.446088 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:09.446058 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" Jul 28 17:44:10.939829 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:10.939789 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6"] Jul 28 17:44:10.943269 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:10.943246 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" Jul 28 17:44:10.945894 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:10.945878 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-e4209-serving-cert\"" Jul 28 17:44:10.945894 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:10.945885 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"switch-graph-e4209-kube-rbac-proxy-sar-config\"" Jul 28 17:44:10.953517 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:10.953496 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6"] Jul 28 17:44:11.026306 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:11.026269 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a57c3967-9b51-4cfa-b639-1c54ceda95fa-openshift-service-ca-bundle\") pod \"switch-graph-e4209-8484bbd765-5lcd6\" (UID: \"a57c3967-9b51-4cfa-b639-1c54ceda95fa\") " pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" Jul 28 17:44:11.026510 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:11.026349 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a57c3967-9b51-4cfa-b639-1c54ceda95fa-proxy-tls\") pod \"switch-graph-e4209-8484bbd765-5lcd6\" (UID: \"a57c3967-9b51-4cfa-b639-1c54ceda95fa\") " pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" Jul 28 17:44:11.127447 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:11.127380 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a57c3967-9b51-4cfa-b639-1c54ceda95fa-proxy-tls\") pod \"switch-graph-e4209-8484bbd765-5lcd6\" (UID: \"a57c3967-9b51-4cfa-b639-1c54ceda95fa\") " pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" Jul 28 17:44:11.127656 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:11.127482 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a57c3967-9b51-4cfa-b639-1c54ceda95fa-openshift-service-ca-bundle\") pod \"switch-graph-e4209-8484bbd765-5lcd6\" (UID: \"a57c3967-9b51-4cfa-b639-1c54ceda95fa\") " pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" Jul 28 17:44:11.127656 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:44:11.127545 2550 secret.go:189] Couldn't get secret kserve-ci-e2e-test/switch-graph-e4209-serving-cert: secret "switch-graph-e4209-serving-cert" not found Jul 28 17:44:11.127656 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:44:11.127635 2550 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a57c3967-9b51-4cfa-b639-1c54ceda95fa-proxy-tls podName:a57c3967-9b51-4cfa-b639-1c54ceda95fa nodeName:}" failed. No retries permitted until 2026-07-28 17:44:11.627612374 +0000 UTC m=+2036.356113704 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/a57c3967-9b51-4cfa-b639-1c54ceda95fa-proxy-tls") pod "switch-graph-e4209-8484bbd765-5lcd6" (UID: "a57c3967-9b51-4cfa-b639-1c54ceda95fa") : secret "switch-graph-e4209-serving-cert" not found Jul 28 17:44:11.128086 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:11.128066 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a57c3967-9b51-4cfa-b639-1c54ceda95fa-openshift-service-ca-bundle\") pod \"switch-graph-e4209-8484bbd765-5lcd6\" (UID: \"a57c3967-9b51-4cfa-b639-1c54ceda95fa\") " pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" Jul 28 17:44:11.630936 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:11.630896 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a57c3967-9b51-4cfa-b639-1c54ceda95fa-proxy-tls\") pod \"switch-graph-e4209-8484bbd765-5lcd6\" (UID: \"a57c3967-9b51-4cfa-b639-1c54ceda95fa\") " pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" Jul 28 17:44:11.633333 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:11.633310 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a57c3967-9b51-4cfa-b639-1c54ceda95fa-proxy-tls\") pod \"switch-graph-e4209-8484bbd765-5lcd6\" (UID: \"a57c3967-9b51-4cfa-b639-1c54ceda95fa\") " pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" Jul 28 17:44:11.853927 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:11.853889 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" Jul 28 17:44:11.983911 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:11.983875 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6"] Jul 28 17:44:11.986916 ip-10-0-134-14 kubenswrapper[2550]: W0728 17:44:11.986889 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda57c3967_9b51_4cfa_b639_1c54ceda95fa.slice/crio-545ae0a94ef4fee8cb8cb4e7a3749cec3c174052a411c729f8f0262244e0ba79 WatchSource:0}: Error finding container 545ae0a94ef4fee8cb8cb4e7a3749cec3c174052a411c729f8f0262244e0ba79: Status 404 returned error can't find the container with id 545ae0a94ef4fee8cb8cb4e7a3749cec3c174052a411c729f8f0262244e0ba79 Jul 28 17:44:12.479340 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:12.479300 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" event={"ID":"a57c3967-9b51-4cfa-b639-1c54ceda95fa","Type":"ContainerStarted","Data":"3d0c92c4fcfa03354a391bc8150890764a690f93c14c5c69cb922432976211c0"} Jul 28 17:44:12.479537 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:12.479380 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" event={"ID":"a57c3967-9b51-4cfa-b639-1c54ceda95fa","Type":"ContainerStarted","Data":"545ae0a94ef4fee8cb8cb4e7a3749cec3c174052a411c729f8f0262244e0ba79"} Jul 28 17:44:12.479537 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:12.479443 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" Jul 28 17:44:12.497804 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:12.497714 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" podStartSLOduration=2.497676749 podStartE2EDuration="2.497676749s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 17:44:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 17:44:12.497335463 +0000 UTC m=+2037.225836789" watchObservedRunningTime="2026-07-28 17:44:12.497676749 +0000 UTC m=+2037.226178078" Jul 28 17:44:18.489894 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:44:18.489866 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" Jul 28 17:52:16.191216 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:16.191178 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv"] Jul 28 17:52:16.191750 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:16.191457 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" podUID="11ad9f23-e3f4-4d26-b401-37dab78becc3" containerName="splitter-graph-4f50a" containerID="cri-o://f665fa69e7ee1502b6b87fd192067cb48275cfc489365a3f3bf3f0914ed9aeea" gracePeriod=30 Jul 28 17:52:19.444084 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:19.444041 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" podUID="11ad9f23-e3f4-4d26-b401-37dab78becc3" containerName="splitter-graph-4f50a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:52:24.444460 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:24.444419 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" podUID="11ad9f23-e3f4-4d26-b401-37dab78becc3" containerName="splitter-graph-4f50a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:52:29.444657 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:29.444610 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" podUID="11ad9f23-e3f4-4d26-b401-37dab78becc3" containerName="splitter-graph-4f50a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:52:29.445044 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:29.444714 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" Jul 28 17:52:34.444716 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:34.444673 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" podUID="11ad9f23-e3f4-4d26-b401-37dab78becc3" containerName="splitter-graph-4f50a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:52:39.444254 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:39.444212 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" podUID="11ad9f23-e3f4-4d26-b401-37dab78becc3" containerName="splitter-graph-4f50a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:52:44.444112 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:44.444068 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" podUID="11ad9f23-e3f4-4d26-b401-37dab78becc3" containerName="splitter-graph-4f50a" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 17:52:46.339103 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:46.339074 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" Jul 28 17:52:46.359927 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:46.359895 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11ad9f23-e3f4-4d26-b401-37dab78becc3-openshift-service-ca-bundle\") pod \"11ad9f23-e3f4-4d26-b401-37dab78becc3\" (UID: \"11ad9f23-e3f4-4d26-b401-37dab78becc3\") " Jul 28 17:52:46.360096 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:46.359965 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/11ad9f23-e3f4-4d26-b401-37dab78becc3-proxy-tls\") pod \"11ad9f23-e3f4-4d26-b401-37dab78becc3\" (UID: \"11ad9f23-e3f4-4d26-b401-37dab78becc3\") " Jul 28 17:52:46.360268 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:46.360241 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11ad9f23-e3f4-4d26-b401-37dab78becc3-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "11ad9f23-e3f4-4d26-b401-37dab78becc3" (UID: "11ad9f23-e3f4-4d26-b401-37dab78becc3"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 17:52:46.362169 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:46.362141 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11ad9f23-e3f4-4d26-b401-37dab78becc3-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "11ad9f23-e3f4-4d26-b401-37dab78becc3" (UID: "11ad9f23-e3f4-4d26-b401-37dab78becc3"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 17:52:46.460628 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:46.460534 2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11ad9f23-e3f4-4d26-b401-37dab78becc3-openshift-service-ca-bundle\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:52:46.460628 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:46.460566 2550 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/11ad9f23-e3f4-4d26-b401-37dab78becc3-proxy-tls\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 17:52:46.622314 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:46.622277 2550 generic.go:356] "Generic (PLEG): container finished" podID="11ad9f23-e3f4-4d26-b401-37dab78becc3" containerID="f665fa69e7ee1502b6b87fd192067cb48275cfc489365a3f3bf3f0914ed9aeea" exitCode=0 Jul 28 17:52:46.622526 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:46.622323 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" Jul 28 17:52:46.622526 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:46.622325 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" event={"ID":"11ad9f23-e3f4-4d26-b401-37dab78becc3","Type":"ContainerDied","Data":"f665fa69e7ee1502b6b87fd192067cb48275cfc489365a3f3bf3f0914ed9aeea"} Jul 28 17:52:46.622526 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:46.622461 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv" event={"ID":"11ad9f23-e3f4-4d26-b401-37dab78becc3","Type":"ContainerDied","Data":"04fce6543d8814faf3cd876a0a98b892feb161e92ad0886e757a1d64505be706"} Jul 28 17:52:46.622526 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:46.622499 2550 scope.go:117] "RemoveContainer" containerID="f665fa69e7ee1502b6b87fd192067cb48275cfc489365a3f3bf3f0914ed9aeea" Jul 28 17:52:46.634443 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:46.634419 2550 scope.go:117] "RemoveContainer" containerID="f665fa69e7ee1502b6b87fd192067cb48275cfc489365a3f3bf3f0914ed9aeea" Jul 28 17:52:46.634772 ip-10-0-134-14 kubenswrapper[2550]: E0728 17:52:46.634749 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f665fa69e7ee1502b6b87fd192067cb48275cfc489365a3f3bf3f0914ed9aeea\": container with ID starting with f665fa69e7ee1502b6b87fd192067cb48275cfc489365a3f3bf3f0914ed9aeea not found: ID does not exist" containerID="f665fa69e7ee1502b6b87fd192067cb48275cfc489365a3f3bf3f0914ed9aeea" Jul 28 17:52:46.634834 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:46.634782 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f665fa69e7ee1502b6b87fd192067cb48275cfc489365a3f3bf3f0914ed9aeea"} err="failed to get container status \"f665fa69e7ee1502b6b87fd192067cb48275cfc489365a3f3bf3f0914ed9aeea\": rpc error: code = NotFound desc = could not find container \"f665fa69e7ee1502b6b87fd192067cb48275cfc489365a3f3bf3f0914ed9aeea\": container with ID starting with f665fa69e7ee1502b6b87fd192067cb48275cfc489365a3f3bf3f0914ed9aeea not found: ID does not exist" Jul 28 17:52:46.645630 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:46.645594 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv"] Jul 28 17:52:46.649307 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:46.649272 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/splitter-graph-4f50a-747967f6f4-dwptv"] Jul 28 17:52:47.771335 ip-10-0-134-14 kubenswrapper[2550]: I0728 17:52:47.771299 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11ad9f23-e3f4-4d26-b401-37dab78becc3" path="/var/lib/kubelet/pods/11ad9f23-e3f4-4d26-b401-37dab78becc3/volumes" Jul 28 18:00:30.244104 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:30.244061 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6"] Jul 28 18:00:30.249120 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:30.244433 2550 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" podUID="a57c3967-9b51-4cfa-b639-1c54ceda95fa" containerName="switch-graph-e4209" containerID="cri-o://3d0c92c4fcfa03354a391bc8150890764a690f93c14c5c69cb922432976211c0" gracePeriod=30 Jul 28 18:00:33.488050 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:33.488013 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" podUID="a57c3967-9b51-4cfa-b639-1c54ceda95fa" containerName="switch-graph-e4209" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 18:00:38.487907 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:38.487867 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" podUID="a57c3967-9b51-4cfa-b639-1c54ceda95fa" containerName="switch-graph-e4209" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 18:00:43.488435 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:43.488377 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" podUID="a57c3967-9b51-4cfa-b639-1c54ceda95fa" containerName="switch-graph-e4209" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 18:00:43.488865 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:43.488574 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" Jul 28 18:00:46.986972 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:46.986939 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-e4209-8484bbd765-5lcd6_a57c3967-9b51-4cfa-b639-1c54ceda95fa/switch-graph-e4209/0.log" Jul 28 18:00:47.853243 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:47.853209 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-e4209-8484bbd765-5lcd6_a57c3967-9b51-4cfa-b639-1c54ceda95fa/switch-graph-e4209/0.log" Jul 28 18:00:48.488093 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:48.488048 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" podUID="a57c3967-9b51-4cfa-b639-1c54ceda95fa" containerName="switch-graph-e4209" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 18:00:48.670330 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:48.670297 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-e4209-8484bbd765-5lcd6_a57c3967-9b51-4cfa-b639-1c54ceda95fa/switch-graph-e4209/0.log" Jul 28 18:00:49.477173 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:49.477141 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-e4209-8484bbd765-5lcd6_a57c3967-9b51-4cfa-b639-1c54ceda95fa/switch-graph-e4209/0.log" Jul 28 18:00:50.280850 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:50.280817 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-e4209-8484bbd765-5lcd6_a57c3967-9b51-4cfa-b639-1c54ceda95fa/switch-graph-e4209/0.log" Jul 28 18:00:51.144947 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:51.144919 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-e4209-8484bbd765-5lcd6_a57c3967-9b51-4cfa-b639-1c54ceda95fa/switch-graph-e4209/0.log" Jul 28 18:00:52.063672 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:52.063639 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-e4209-8484bbd765-5lcd6_a57c3967-9b51-4cfa-b639-1c54ceda95fa/switch-graph-e4209/0.log" Jul 28 18:00:52.889431 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:52.889383 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-e4209-8484bbd765-5lcd6_a57c3967-9b51-4cfa-b639-1c54ceda95fa/switch-graph-e4209/0.log" Jul 28 18:00:53.487509 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:53.487470 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" podUID="a57c3967-9b51-4cfa-b639-1c54ceda95fa" containerName="switch-graph-e4209" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 18:00:53.679799 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:53.679770 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-e4209-8484bbd765-5lcd6_a57c3967-9b51-4cfa-b639-1c54ceda95fa/switch-graph-e4209/0.log" Jul 28 18:00:54.479173 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:54.479147 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-e4209-8484bbd765-5lcd6_a57c3967-9b51-4cfa-b639-1c54ceda95fa/switch-graph-e4209/0.log" Jul 28 18:00:55.251206 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:55.251173 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-e4209-8484bbd765-5lcd6_a57c3967-9b51-4cfa-b639-1c54ceda95fa/switch-graph-e4209/0.log" Jul 28 18:00:56.124667 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:56.124631 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-e4209-8484bbd765-5lcd6_a57c3967-9b51-4cfa-b639-1c54ceda95fa/switch-graph-e4209/0.log" Jul 28 18:00:56.941772 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:56.941720 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-e4209-8484bbd765-5lcd6_a57c3967-9b51-4cfa-b639-1c54ceda95fa/switch-graph-e4209/0.log" Jul 28 18:00:57.747020 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:57.746991 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_switch-graph-e4209-8484bbd765-5lcd6_a57c3967-9b51-4cfa-b639-1c54ceda95fa/switch-graph-e4209/0.log" Jul 28 18:00:58.487963 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:00:58.487924 2550 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" podUID="a57c3967-9b51-4cfa-b639-1c54ceda95fa" containerName="switch-graph-e4209" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 18:01:00.394898 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:00.394875 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" Jul 28 18:01:00.509530 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:00.509425 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a57c3967-9b51-4cfa-b639-1c54ceda95fa-openshift-service-ca-bundle\") pod \"a57c3967-9b51-4cfa-b639-1c54ceda95fa\" (UID: \"a57c3967-9b51-4cfa-b639-1c54ceda95fa\") " Jul 28 18:01:00.509530 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:00.509519 2550 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a57c3967-9b51-4cfa-b639-1c54ceda95fa-proxy-tls\") pod \"a57c3967-9b51-4cfa-b639-1c54ceda95fa\" (UID: \"a57c3967-9b51-4cfa-b639-1c54ceda95fa\") " Jul 28 18:01:00.509854 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:00.509826 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a57c3967-9b51-4cfa-b639-1c54ceda95fa-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "a57c3967-9b51-4cfa-b639-1c54ceda95fa" (UID: "a57c3967-9b51-4cfa-b639-1c54ceda95fa"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 18:01:00.511591 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:00.511568 2550 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a57c3967-9b51-4cfa-b639-1c54ceda95fa-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "a57c3967-9b51-4cfa-b639-1c54ceda95fa" (UID: "a57c3967-9b51-4cfa-b639-1c54ceda95fa"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 18:01:00.610488 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:00.610450 2550 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a57c3967-9b51-4cfa-b639-1c54ceda95fa-proxy-tls\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 18:01:00.610488 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:00.610482 2550 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a57c3967-9b51-4cfa-b639-1c54ceda95fa-openshift-service-ca-bundle\") on node \"ip-10-0-134-14.ec2.internal\" DevicePath \"\"" Jul 28 18:01:00.678143 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:00.678108 2550 generic.go:356] "Generic (PLEG): container finished" podID="a57c3967-9b51-4cfa-b639-1c54ceda95fa" containerID="3d0c92c4fcfa03354a391bc8150890764a690f93c14c5c69cb922432976211c0" exitCode=0 Jul 28 18:01:00.678326 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:00.678153 2550 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" Jul 28 18:01:00.678326 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:00.678202 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" event={"ID":"a57c3967-9b51-4cfa-b639-1c54ceda95fa","Type":"ContainerDied","Data":"3d0c92c4fcfa03354a391bc8150890764a690f93c14c5c69cb922432976211c0"} Jul 28 18:01:00.678326 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:00.678258 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6" event={"ID":"a57c3967-9b51-4cfa-b639-1c54ceda95fa","Type":"ContainerDied","Data":"545ae0a94ef4fee8cb8cb4e7a3749cec3c174052a411c729f8f0262244e0ba79"} Jul 28 18:01:00.678326 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:00.678286 2550 scope.go:117] "RemoveContainer" containerID="3d0c92c4fcfa03354a391bc8150890764a690f93c14c5c69cb922432976211c0" Jul 28 18:01:00.686620 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:00.686603 2550 scope.go:117] "RemoveContainer" containerID="3d0c92c4fcfa03354a391bc8150890764a690f93c14c5c69cb922432976211c0" Jul 28 18:01:00.686869 ip-10-0-134-14 kubenswrapper[2550]: E0728 18:01:00.686849 2550 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d0c92c4fcfa03354a391bc8150890764a690f93c14c5c69cb922432976211c0\": container with ID starting with 3d0c92c4fcfa03354a391bc8150890764a690f93c14c5c69cb922432976211c0 not found: ID does not exist" containerID="3d0c92c4fcfa03354a391bc8150890764a690f93c14c5c69cb922432976211c0" Jul 28 18:01:00.686934 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:00.686880 2550 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d0c92c4fcfa03354a391bc8150890764a690f93c14c5c69cb922432976211c0"} err="failed to get container status \"3d0c92c4fcfa03354a391bc8150890764a690f93c14c5c69cb922432976211c0\": rpc error: code = NotFound desc = could not find container \"3d0c92c4fcfa03354a391bc8150890764a690f93c14c5c69cb922432976211c0\": container with ID starting with 3d0c92c4fcfa03354a391bc8150890764a690f93c14c5c69cb922432976211c0 not found: ID does not exist" Jul 28 18:01:00.705689 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:00.705659 2550 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6"] Jul 28 18:01:00.708381 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:00.708358 2550 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/switch-graph-e4209-8484bbd765-5lcd6"] Jul 28 18:01:01.771470 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:01.771438 2550 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a57c3967-9b51-4cfa-b639-1c54ceda95fa" path="/var/lib/kubelet/pods/a57c3967-9b51-4cfa-b639-1c54ceda95fa/volumes" Jul 28 18:01:05.452074 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:05.452045 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gzjsf_143142e3-d09f-4dde-bf59-1fffafc3e2fc/global-pull-secret-syncer/0.log" Jul 28 18:01:05.629901 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:05.629868 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-g6hlp_6457cb80-cd0c-4342-8fab-4ad86a56af24/konnectivity-agent/0.log" Jul 28 18:01:05.698792 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:05.698759 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-14.ec2.internal_7cc46c224c17195fb5410e94fda8e4a3/haproxy/0.log" Jul 28 18:01:09.896007 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:09.895918 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-ggldr_f953c640-f9e2-4f72-971f-4719d48a19ef/cluster-monitoring-operator/0.log" Jul 28 18:01:10.129682 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:10.129656 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kc7cf_e30b4440-4d5f-4bc1-91ed-b67bc5af1aad/node-exporter/0.log" Jul 28 18:01:10.165941 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:10.165867 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kc7cf_e30b4440-4d5f-4bc1-91ed-b67bc5af1aad/kube-rbac-proxy/0.log" Jul 28 18:01:10.199281 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:10.199258 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-kc7cf_e30b4440-4d5f-4bc1-91ed-b67bc5af1aad/init-textfile/0.log" Jul 28 18:01:10.954947 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:10.954920 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f67645548-v65s7_7e8c42df-5eea-48e7-bf00-15364556b04b/thanos-query/0.log" Jul 28 18:01:10.988407 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:10.988364 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f67645548-v65s7_7e8c42df-5eea-48e7-bf00-15364556b04b/kube-rbac-proxy-web/0.log" Jul 28 18:01:11.021712 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:11.021686 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f67645548-v65s7_7e8c42df-5eea-48e7-bf00-15364556b04b/kube-rbac-proxy/0.log" Jul 28 18:01:11.049966 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:11.049945 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f67645548-v65s7_7e8c42df-5eea-48e7-bf00-15364556b04b/prom-label-proxy/0.log" Jul 28 18:01:11.077216 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:11.077195 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f67645548-v65s7_7e8c42df-5eea-48e7-bf00-15364556b04b/kube-rbac-proxy-rules/0.log" Jul 28 18:01:11.104232 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:11.104212 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-f67645548-v65s7_7e8c42df-5eea-48e7-bf00-15364556b04b/kube-rbac-proxy-metrics/0.log" Jul 28 18:01:12.543345 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:12.543319 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-9whj6_8cc0aa75-c527-429b-9acd-37b1b1f9c6c7/networking-console-plugin/0.log" Jul 28 18:01:13.456531 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.456495 2550 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p"] Jul 28 18:01:13.456850 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.456837 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11ad9f23-e3f4-4d26-b401-37dab78becc3" containerName="splitter-graph-4f50a" Jul 28 18:01:13.456896 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.456852 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="11ad9f23-e3f4-4d26-b401-37dab78becc3" containerName="splitter-graph-4f50a" Jul 28 18:01:13.456896 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.456872 2550 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a57c3967-9b51-4cfa-b639-1c54ceda95fa" containerName="switch-graph-e4209" Jul 28 18:01:13.456896 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.456883 2550 state_mem.go:107] "Deleted CPUSet assignment" podUID="a57c3967-9b51-4cfa-b639-1c54ceda95fa" containerName="switch-graph-e4209" Jul 28 18:01:13.457004 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.456972 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="a57c3967-9b51-4cfa-b639-1c54ceda95fa" containerName="switch-graph-e4209" Jul 28 18:01:13.457004 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.456982 2550 memory_manager.go:368] "RemoveStaleState removing state" podUID="11ad9f23-e3f4-4d26-b401-37dab78becc3" containerName="splitter-graph-4f50a" Jul 28 18:01:13.461507 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.461485 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" Jul 28 18:01:13.464955 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.464933 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ck55s\"/\"openshift-service-ca.crt\"" Jul 28 18:01:13.465112 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.464937 2550 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ck55s\"/\"kube-root-ca.crt\"" Jul 28 18:01:13.466155 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.466137 2550 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ck55s\"/\"default-dockercfg-p6qwz\"" Jul 28 18:01:13.471986 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.471950 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p"] Jul 28 18:01:13.500720 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.500694 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6dfb94bd9f-8tzpc_133915b8-a27b-40b7-b3be-5b0ad5975341/console/0.log" Jul 28 18:01:13.614190 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.614154 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qhxz\" (UniqueName: \"kubernetes.io/projected/ffdb3614-e410-4765-bc0a-6fab0ef15299-kube-api-access-9qhxz\") pod \"perf-node-gather-daemonset-5vg9p\" (UID: \"ffdb3614-e410-4765-bc0a-6fab0ef15299\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" Jul 28 18:01:13.614615 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.614204 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ffdb3614-e410-4765-bc0a-6fab0ef15299-sys\") pod \"perf-node-gather-daemonset-5vg9p\" (UID: \"ffdb3614-e410-4765-bc0a-6fab0ef15299\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" Jul 28 18:01:13.614615 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.614272 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ffdb3614-e410-4765-bc0a-6fab0ef15299-lib-modules\") pod \"perf-node-gather-daemonset-5vg9p\" (UID: \"ffdb3614-e410-4765-bc0a-6fab0ef15299\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" Jul 28 18:01:13.614615 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.614288 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ffdb3614-e410-4765-bc0a-6fab0ef15299-proc\") pod \"perf-node-gather-daemonset-5vg9p\" (UID: \"ffdb3614-e410-4765-bc0a-6fab0ef15299\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" Jul 28 18:01:13.614615 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.614323 2550 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ffdb3614-e410-4765-bc0a-6fab0ef15299-podres\") pod \"perf-node-gather-daemonset-5vg9p\" (UID: \"ffdb3614-e410-4765-bc0a-6fab0ef15299\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" Jul 28 18:01:13.715576 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.715478 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ffdb3614-e410-4765-bc0a-6fab0ef15299-sys\") pod \"perf-node-gather-daemonset-5vg9p\" (UID: \"ffdb3614-e410-4765-bc0a-6fab0ef15299\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" Jul 28 18:01:13.715699 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.715624 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ffdb3614-e410-4765-bc0a-6fab0ef15299-sys\") pod \"perf-node-gather-daemonset-5vg9p\" (UID: \"ffdb3614-e410-4765-bc0a-6fab0ef15299\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" Jul 28 18:01:13.715699 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.715642 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ffdb3614-e410-4765-bc0a-6fab0ef15299-lib-modules\") pod \"perf-node-gather-daemonset-5vg9p\" (UID: \"ffdb3614-e410-4765-bc0a-6fab0ef15299\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" Jul 28 18:01:13.715699 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.715684 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ffdb3614-e410-4765-bc0a-6fab0ef15299-proc\") pod \"perf-node-gather-daemonset-5vg9p\" (UID: \"ffdb3614-e410-4765-bc0a-6fab0ef15299\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" Jul 28 18:01:13.715799 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.715723 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ffdb3614-e410-4765-bc0a-6fab0ef15299-podres\") pod \"perf-node-gather-daemonset-5vg9p\" (UID: \"ffdb3614-e410-4765-bc0a-6fab0ef15299\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" Jul 28 18:01:13.715799 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.715757 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ffdb3614-e410-4765-bc0a-6fab0ef15299-lib-modules\") pod \"perf-node-gather-daemonset-5vg9p\" (UID: \"ffdb3614-e410-4765-bc0a-6fab0ef15299\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" Jul 28 18:01:13.715799 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.715765 2550 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qhxz\" (UniqueName: \"kubernetes.io/projected/ffdb3614-e410-4765-bc0a-6fab0ef15299-kube-api-access-9qhxz\") pod \"perf-node-gather-daemonset-5vg9p\" (UID: \"ffdb3614-e410-4765-bc0a-6fab0ef15299\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" Jul 28 18:01:13.715901 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.715803 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ffdb3614-e410-4765-bc0a-6fab0ef15299-proc\") pod \"perf-node-gather-daemonset-5vg9p\" (UID: \"ffdb3614-e410-4765-bc0a-6fab0ef15299\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" Jul 28 18:01:13.715901 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.715866 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ffdb3614-e410-4765-bc0a-6fab0ef15299-podres\") pod \"perf-node-gather-daemonset-5vg9p\" (UID: \"ffdb3614-e410-4765-bc0a-6fab0ef15299\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" Jul 28 18:01:13.725016 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.724989 2550 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qhxz\" (UniqueName: \"kubernetes.io/projected/ffdb3614-e410-4765-bc0a-6fab0ef15299-kube-api-access-9qhxz\") pod \"perf-node-gather-daemonset-5vg9p\" (UID: \"ffdb3614-e410-4765-bc0a-6fab0ef15299\") " pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" Jul 28 18:01:13.777959 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.777931 2550 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" Jul 28 18:01:13.908054 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:13.908026 2550 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p"] Jul 28 18:01:13.910232 ip-10-0-134-14 kubenswrapper[2550]: W0728 18:01:13.910202 2550 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podffdb3614_e410_4765_bc0a_6fab0ef15299.slice/crio-66dbc597051e21d714766822804d8b92a2bbf7e93303ac4d04fdc8837b814fed WatchSource:0}: Error finding container 66dbc597051e21d714766822804d8b92a2bbf7e93303ac4d04fdc8837b814fed: Status 404 returned error can't find the container with id 66dbc597051e21d714766822804d8b92a2bbf7e93303ac4d04fdc8837b814fed Jul 28 18:01:14.738347 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:14.738309 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" event={"ID":"ffdb3614-e410-4765-bc0a-6fab0ef15299","Type":"ContainerStarted","Data":"42567e8e95b99842334a00a267d73dc027b6b8f29f523c45241905f5e7c12157"} Jul 28 18:01:14.738829 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:14.738362 2550 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" event={"ID":"ffdb3614-e410-4765-bc0a-6fab0ef15299","Type":"ContainerStarted","Data":"66dbc597051e21d714766822804d8b92a2bbf7e93303ac4d04fdc8837b814fed"} Jul 28 18:01:14.738829 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:14.738415 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" Jul 28 18:01:14.759316 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:14.759254 2550 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" podStartSLOduration=1.759237774 podStartE2EDuration="1.759237774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 18:01:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 18:01:14.757474316 +0000 UTC m=+3059.485975653" watchObservedRunningTime="2026-07-28 18:01:14.759237774 +0000 UTC m=+3059.487739152" Jul 28 18:01:15.143566 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:15.143537 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jx7cl_4b6d4302-fb6f-44e7-8b39-07a78c35b277/dns/0.log" Jul 28 18:01:15.181425 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:15.181377 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jx7cl_4b6d4302-fb6f-44e7-8b39-07a78c35b277/kube-rbac-proxy/0.log" Jul 28 18:01:15.280849 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:15.280821 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vpsrj_56e96be9-8647-407e-b2d9-f6718d894258/dns-node-resolver/0.log" Jul 28 18:01:15.757601 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:15.757570 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jblj2_c05afc7b-dee8-4de5-956e-336b79ac9867/node-ca/0.log" Jul 28 18:01:16.980914 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:16.980883 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fxdhd_6f52b7f0-cbf9-46aa-bac6-b7fd7486ea50/serve-healthcheck-canary/0.log" Jul 28 18:01:17.606004 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:17.605976 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xg5vg_8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696/kube-rbac-proxy/0.log" Jul 28 18:01:17.632788 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:17.632759 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xg5vg_8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696/exporter/0.log" Jul 28 18:01:17.660623 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:17.660596 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xg5vg_8bce64c0-eeb2-43b9-aca7-ed3b7bbc7696/extractor/0.log" Jul 28 18:01:19.912639 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:19.912605 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-654c557856-qkb7v_6fa97b7e-3a9a-4769-998a-40ec69ac0562/manager/0.log" Jul 28 18:01:20.756119 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:20.756091 2550 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ck55s/perf-node-gather-daemonset-5vg9p" Jul 28 18:01:24.974236 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:24.974181 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp_73e0baae-ba8a-4ef3-aff6-a31782400af6/kube-storage-version-migrator-operator/1.log" Jul 28 18:01:24.975061 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:24.975041 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-lwwjp_73e0baae-ba8a-4ef3-aff6-a31782400af6/kube-storage-version-migrator-operator/0.log" Jul 28 18:01:26.003827 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:26.003793 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4428b_1dca65a6-b651-46fd-a9ef-fc4eb8ff667e/kube-multus/0.log" Jul 28 18:01:26.037782 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:26.037754 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qslrz_d9ae751d-09fa-4166-b47b-0e069281d796/kube-multus-additional-cni-plugins/0.log" Jul 28 18:01:26.067321 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:26.067294 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qslrz_d9ae751d-09fa-4166-b47b-0e069281d796/egress-router-binary-copy/0.log" Jul 28 18:01:26.094808 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:26.094776 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qslrz_d9ae751d-09fa-4166-b47b-0e069281d796/cni-plugins/0.log" Jul 28 18:01:26.129070 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:26.129039 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qslrz_d9ae751d-09fa-4166-b47b-0e069281d796/bond-cni-plugin/0.log" Jul 28 18:01:26.168096 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:26.168063 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qslrz_d9ae751d-09fa-4166-b47b-0e069281d796/routeoverride-cni/0.log" Jul 28 18:01:26.198347 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:26.198320 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qslrz_d9ae751d-09fa-4166-b47b-0e069281d796/whereabouts-cni-bincopy/0.log" Jul 28 18:01:26.226074 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:26.226040 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-qslrz_d9ae751d-09fa-4166-b47b-0e069281d796/whereabouts-cni/0.log" Jul 28 18:01:26.867081 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:26.867054 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4l6cw_6cb7d969-3477-425b-9bf1-3192c9d7d94c/network-metrics-daemon/0.log" Jul 28 18:01:26.914681 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:26.914650 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4l6cw_6cb7d969-3477-425b-9bf1-3192c9d7d94c/kube-rbac-proxy/0.log" Jul 28 18:01:27.915930 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:27.915903 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4p62f_ed8e1716-5cda-4a93-b714-15d88545f8bd/ovn-controller/0.log" Jul 28 18:01:27.956416 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:27.956379 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4p62f_ed8e1716-5cda-4a93-b714-15d88545f8bd/ovn-acl-logging/0.log" Jul 28 18:01:27.995879 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:27.995854 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4p62f_ed8e1716-5cda-4a93-b714-15d88545f8bd/kube-rbac-proxy-node/0.log" Jul 28 18:01:28.036158 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:28.036134 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4p62f_ed8e1716-5cda-4a93-b714-15d88545f8bd/kube-rbac-proxy-ovn-metrics/0.log" Jul 28 18:01:28.074534 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:28.074504 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4p62f_ed8e1716-5cda-4a93-b714-15d88545f8bd/northd/0.log" Jul 28 18:01:28.125221 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:28.125196 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4p62f_ed8e1716-5cda-4a93-b714-15d88545f8bd/nbdb/0.log" Jul 28 18:01:28.166603 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:28.166522 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4p62f_ed8e1716-5cda-4a93-b714-15d88545f8bd/sbdb/0.log" Jul 28 18:01:28.232558 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:28.232527 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4p62f_ed8e1716-5cda-4a93-b714-15d88545f8bd/ovnkube-controller/0.log" Jul 28 18:01:30.139637 ip-10-0-134-14 kubenswrapper[2550]: I0728 18:01:30.139603 2550 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-7d88cb55d8-xv5f2_f16ab6fd-caec-4bb9-8aa8-d146b004e239/check-endpoints/0.log"